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Summary 


This  report  deals  with  the  effect  of  clipping  on  the  performance  of 
an  active  sonar  system  using  conventional  beanforning  techniques  followed 
by  replica  correlation.  Detection  as  well  as  range  and  Doppler  estimation 
are  considered. 

Two  basic  assumptions  are  made  throughout  the  analysis: 

a)  The  noise  field  (ambient  or  reverberation)  is  Gaussian  and  has 
the  same  power  level  at  each  hydrophone. 

b)  The  input  signal-to-noise  ratio  at  each  hydrophone  is  small. 

In  addition  much,  though  not  all,  of  the  work  assumes  a  transmitted 
signal  narrow  in  bandwidth  compared  with  its  center  frequency.  The  model 
for  reverberation  noise  postulates  a  scries  of  stationary,  Poisson  distributed 
scattering  centers.  The  array  geometry  is  quite  arbitrary;  but  certain 
computations  require  beam  patterns  narrow  enough  to  permit  approximations  of 
the  form,  sin  0  =  0  over  the  effective  dimensions  of  the  pattern. 

Tie  quantity  of  primary  interest  is  the  clipping  loss  R  ,  defined  as 
the  output  signal-to-noise  ratio  of  the  clipped  instrumentation  divided  by 
thi.  output  sigml-to-noise  ratio  of  the  unclipped  (but  otherwise  identical) 
instrumentation . 

The  following  results  are  obtained  for  detection: 

1)  If  the  noise  (ambient  or  reverberation)  is  independent  from  hydrophone 
to  hydrophone  one  can  demonstrate  with  complete  generality  that 
R  £  1  .  One  can  further  demonstrate  that,  without  additional 
restrictions  on  signal  and  noise,  a  lower  bound  of  R  -  0  can  be 
approached  arbitrarily  closely.  To  set  meaningful  bounds  on  clipping 
loss  one  must  therefore  restrict  the  class  of  admissible  signals  ind 
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noises.  A  practically  realistic  and  analytically  fruitful 
restriction  is  the  assumption  of  narrow-band  signals,  which  under¬ 
lies  all  remaining  results. 

2)  If  the  noise  (ambient  or  reverberation)  is  independent  from 
hydrophone  to  hydrophone  and  if  signal  and  noise  are  confined  to 
the  same  frequency  band  (narrow  compared  with  the  center  frequency) 
the  clipping  loss  is  bounded  by  0.89  £  R  <  i  . 

3)  If  the  noise  does  not  fall  into  the  same  frequency  band  as  the 
signal,  large  clipping  losses  can  occur.  This  is  practically 
important  in  a  reverberation  limited  environment  when  the  target 
return  is  subject  to  large  Doppler  shifts.  In  such  situations  R 
can  approach  arbitrarily  close  to  zero  if  the  Doppler  shift  is 
large  enough. 

A)  If  statistical  dependences  are  allowed  between  the  noise  at  different 
hydrophones,  one  requires  further  restrictions  before  useful  lower 
bounds  can  be  set  on  R  .  An  example  is  worked  cut  to  demonstrate 
that,  even  with  purely  isotropic  ambient  , else,  values  of  R 
appreciably  below  0.89  may  be  obtained.  However,  the  example 
requires  such  careful  matching  of  array  geometry  with  carrier 
wavelength  that  it  appears  to  fall  more  into  the  category  of 
analytical  pathologies  than  into  that  of  practically  important 
situations.  A  search  (by  no  means  exhaustive)  for  more  realistic 
examples  in  which  ambient  noise  would  produce  values  of  R  below 
0.89  led  to  negative  results. 

5)  Probably  the  most  important  situation  from  a  practical  point,  of 


view  is  that  of  a  reverberation  limited  environment.  It  was 


therefore  studied  in  some  detail.  The  most  useful  results  were 


obtained  under  the  assumption  that  the  array  dimensions  are  small 
compared  with  the  wavelength  of  the  highest  modulating  frequency, 
(i.e.,  the  wavelength  of  the  maximum  frequency  deviation  from  the 
carrier) .  The  effect  of  this  assumption  is  to  permit  complete 
separation  of  spatial  and  temporal  effects  in  the  reverberation.  In 
the  absenc  of  target  Doppler  shifts  one  can  then  establish  with 
considerable  generality  that  R  >  0.89  .  In  the  presence  of  target 
Doppler  shifts  one  has,  of  course,  the  phenomenon  discussed  in  3). 
The  key  assumption  concerning  array  dimensions  can  be  weakened 
considerably  if  the  beam  pattern  is  narrow. 

When  one  considers  range  and  Doppler  measurements  one  finds, 
not  surprisingly,  that  the  exact  clipping  loss  depends  to  some 
extent  on  the  specific  instrumentation.  It  is  therefore  not  possible 
to  draw  conclusions  of  quite  the  same  generality  as  in  the  analysis 
of  detection.  Range  (Doppler  shift)  is  measured  by  cross-correlating 
tilt  target  return  with  a  replica  of  the  transmitted  signal  and 
".locating"  th.e  resulting  correlation  function  in  time  (  rcquency)  . 

Dt  ffe.re;it  instrumentations  result  from  different  functional  defini¬ 
tions  of  the  term  "location".  It  appears  reasonable  to  speculate  - 
and  several  sample  computations  tend  to  confirm  this  -  that  most 
teasmabl-  definitions  of  "location"  would  lead  to  rather  similar 
instrument ntions ,  and  in  particular  to  instrumentations  with  very 
similar  sensitivity  to  clipping.  This  report  concerns  itself 
primarily  with  range-  '’Doppler)  measurement  in  a  reverberation  limited 
environment.  Range  (Doppler  shift)  is  measured  by  comparing 


cross-correlations  with  two  replicas  of  slightly  different  delay 
(frequency)  .  Dcppler  shift  is  assumed  known  during  the  range 
measurement  and  range  during  the  Doppler  measurement.  Clipping 
loss  is  defined  as  the  ratio  of  the  rns  range  (Doppler)  errors 
with  and  without  clipping.  The  results  are 

6)  Under  the  conditions  specified  in  5)  the  clipping  loss  factor 
for  range  measurement  has  a  lower  bound  not  significantly  different 
from  the  figure  of  0.89  obtained  for  detection. 

7)  As  night  be  anticipated  from  3)  ,  the  clipping  loss  in  Doppler 
measurement  depends  heavily  on  the  target  Doppler  shift.  Separating 
out  this  tff«-ct  by  working  with  zero  target  Doppler  shift,  cne  can 
again  show  that  the  clipping  loss  factor  has  a  lower  bound  close  to 
0.89  . 

Combining  all  of  the  above,  one  Is  lead  to  the  following  general 
conclusion-  Serious  clipping  losses  arise  in  a  reverberation  limited 
environment  when  the  target  Doppler  shift  is  large  enough  to  move  the  target 
return  largely  out  of  the  reverberation  band.  In  most  other  practically 
interesting  sit.ations  the  clipping  loss  in  detection,  ranging  and  Doppler 
estimation  is  limited  to  a  factor  of  the  order  of  0.89  ,  equivalent  to  about 
1  db  of  input  signal  to-noise  ratio. 
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I.  Introduction 


This  report  is  concerned  with  the  effect  of  clipping  on  the  performance 
of  an  active  sonar  array.  Correlation  with  a  replica  (delayed  and  Doppler 
shifted)  of  the  transmitted  signal  is  used  as  the  basic  signal  processing 
technique . 

The  general  block  diagram  for  detection  is  shown  in  Figure  1.  The 
array  geometry  is  entirely  general. 


s 


Figure  1 

The  delays  bring  the  signal  components  of  all  hydrophone  outputs  into 

alignment .  The  clippers^  then  generate  a  set  of  signals  (x^(t)}  given 

'''In  practice  ttie  clippers  would  probably  precede  the  delays  (which  would 
then  be  digital).  It  is  clear  from  physical  reasoning  that  this  interchange 
would  not  alter  the  x.(t)  . 
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by 


I  1  if  s(t)+n  (t-T  )  >  0 

x.(t)  =  s£njs(t)+n. (t-T  )J  =  l  1  (1) 

1  1  0  if  s(t)-ni(c-Ti)  <  0 

V 

The  x^(t)  are  summed  and  multiplied  by  a  suitable  replica  of  s (  : )  . 

The  resulting  y(t)  is  finally  smoothed  over  n  Deriod  T  comparable  to 
the  duration  of  the  signal.  If  the  noise  has  zero  mean  the  average  out¬ 
put  z  is  zero  in  tie  ubsenct  of  a  signal  ccnpcnent  at  the  hydrophones. 

When  a  signal  component  is  present  ^t  the  hydrophones  z  will  differ  from 
zero,  hence,  if  the  output  z(T)  at  tire  t  exceeds  a  nreset  threshold 
(depending  on  signal-tc-noise  ratio  and  allowed  falsa  alarm  rate)  one  con¬ 
cludes  that  a  target  is  indeed  present. 

To  measure  range  (or  Doppler)  one  night  employ  two  replicas  with 
slightly  different  delays  (cr  Doppler  shifts),  multiply  each  by  the  clipped 
and  summed  hydrophone  cutouts  (x)  and  use  tlu.  difference  between  the 
resulting  y's  as  a  measure  of  range  (or  Doppler  shift).  A  possible 
implementation  is  discussed  in  somewhat  more  d. tail  at  a  later  ncint. 

The  n.  isc  field  is  assumed  to  be  Gaussian.  Thin  is  case’. able  even 
in  a  reverberation  limited  environment  as  lent’  as  nc  major  portion  of 
the  noise  power  is  contributed  by  large  sc.atterers  (false  targets)  .  [See 
Report  Mo .  27  I  . 
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1 I.  General  Relations  for  Detection 

The  output  y(t)  of  the  multiplier  can  be  written  in  the  form 


M 


y ( t)  =  \  x^t.)  s(t)  =  ^  sgn[s(t)+ni(t-ii)]  a(t) 


(2) 


i=l 


i=l 


The  delay  of  the  ’'epl  ic.a  is  here  assumed  to  be  perfectly  matched  to  the 
signal  delay. 

Tlie  mean  value  of  the  detector  output  is 


M_  T 

\  1 


=  '  —  j  s(t)  sgn[s(t)+ni(t-Ti)J  dt 


(3) 


1=1  0 


N  ow 


i(L)+n  (t-i  ^  }  =  Pr  {  [s(t)+n^(t-T^)]  >  0}  -  Pr{  [s(t)+n^(t-T^)]  <•  0}  (4) 


;'i  zero-mean  Gauss  la''  n^(t) 


Pr  1  [s(0+n.  (t-T  .)  ]  >  0}  =  |  e 

11  /2rrN  j0 


_  Ln-s(t)]Z 


2N 


dn  =  h 


1+erf  ■2-^ 
■/2N  J 


(5) 


wile  re 


rl  v 


-1  j 

Jo 


dx 


(6) 


ami  N  is  the  average  noise  power, 
il imi  larly 


l’r{  js(t)tn.  (t  -t  )  ]  <  0}  =  — — 

1  ‘  /2ttN 


o  _  Ln-  s-(.  t  U z 


2N 


dn  =  k 


1-erf 


s(t) 
/2N  J 


(7) 
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It  follows  that 


(O 


sgn[s(t)+n .  (t-r  )]  =  erf 

1  1  /2N 


(8) 


when  max  s(t)  <<  /2N 
t 


sgn[s(t)+n  (t--  > j  . J 


2  sit). 

r  /;T 


(9) 


Hence 


^  j  dt  s?(t) 


for  max  s(t)  <<••  /2N 
t 


ao) 


For  a  figure  of  merit  of  the  detector  we  choose  as  usual  the  output 
signii-to-noi se  ratio,  i.e.,  the  average  output  due  to  the  signal  divided 
by  the  rrns  value  of  the  output  fluctuation.  If  the  input  signal-to-noise 
ratio  is  low,  the  output  fluctuation  is  very  largely  due  to  the  noise 
component  of  the  input,  so  that 


2 

z 


(t) 


MM  T  J _ 

/  }  ~J  j  dt  '  dX  s(t)  s ( X )  sgn | s ( t )+n  ( t-i  )J  sgn[s(X)+n  (A-x.)J 

— 1  — 1  t  ■  J  /  t  1  J  J 

i=i  j-i  o  u 


M  M  T _ 

-  ~~2  )  j  dt  s(t)  )  sgn |n^  ( t-T j )  j  sgn  [n^  0. -t j ) ]  (11) 

T  1=1  j=i  o  A 


From  a  well-known  result  In  noise  theory 


s8n(_n1  (t)  |  sgn  |  n  ^  (A  )  |  =  ~  sin  1  t  ( t -A)  ] 


(12) 


where  (t)  is  the  normalized  cross-correlation  function  of  the  noise 


received  at  the  ith  nd  j'’1’  hydrophoru 


Hence,  for  low  input  signal- 


A-- 


tu-noise  ratio 


:•  ‘ft)  = 


M  h  T  T 

*—  -  dt  s(t)  |  dX  s(X)  —  sin  1 1 p .  (t-X-T,+T  ) 

T  — 1  J  J  "  L  ij  1  J 

i=  1  1*1  0  0 


(13) 


From  Equations  (10)  and  (13)  the  desired  output  signal-to-noise  ratio 


.1  p 


m  = 

^  *0  clipped 


s~(t)  dt 


(1A) 


J 


I  f  MM 

/~  j  dt  s(t)  j  dX  s(A)  ^  ^  r  sin  *  p 4  ( t-X-T^  !  ) 


0 


0 


1-1  j-1 


'ij'  1  -j' 


1 1 1  the  absence  of  clipping 


K  l  2,  ,  d 

J  j  S  (t)  dt 


(15) 


and 


M  M  T 

1_  -  "  \~ ’  ( 

1  1=1  j-1  0 


dr  s(t)  /  dX  s(X)  N , .  .  .  .  (t-X-x ,+t  . ) 


0 


ij  ij  i  j' 


(16) 


N  t  (t-Al  =  n  (t)  n  (X) 

ij  ij  i  j 


(17) 


Hi.ice,  the  output  sign  j 1-to-noise  ratio  in  the  unclipped  case  is 


i'  undipped 


T 

M  f 
T  J 
0 


dt 


/  i  ' 
/K  I  ct 

C  I 

I  J 


T  MM" 

s(t)  J  dX  s(X)  )  /  (t-X-r^Tj ) 

0  i =1  j=l 


(18) 


A— J) 


The  present  atuM)  is  concerned  with  performance  degradation  due  to  clipping. 
Hence  the  ratio  R  of  Equations  (.14)  and  (18)  serves  as  a  convenient  cri¬ 
terion  . 


(S/N)q  clipped 
(S/N)(J  unclipped 


j  T  J  M  _M 

j  j  It  t(.t)  ,  d  ‘.  s  ( X )  Pij  (t-X-Tj+Tj) 

1  j) _  _0 _ i=i  J  =  1  _ 

I  T  I  MM 


it  (t)  '  d •  s(> )  ^  )  N  sin 

>'  i=L  j=l 


Pij(t-X-Ti+,j\ 


(19) 


if  the  average  noi.se  pow.r  is  the  same  at  each  hydrophone,  then 
N,.  =  N  and  Equation  (19)  reduces  to 


K  (  t  ) 

T 

j  li>  !i<» 

M 

>' 

M 

•  .  .  ( t  —  X -  ;  +t  . ) 

1 1  i  J 

/  0 

0 

l--l 

.i-i 

/  T 

r 

M 

M 

1  '  “ 

s(t) 

/  d \  six) 

/  sln"1|pJ1(t-X-TJ,+Tj) 

J  o 

0 

i~l 

j  =  i 

(20) 


Ill  Noise  Independent:  from  Hydrophone  to  Hydrophone 


Aii  Important  special  case  is  that  of  noise  independent  from  hydrophone 
n  that  situation,  Equation  (20)  reduces  to 


I  T  T 

1  r 


j  1  dt  s(t) 

!  "o 

1  dX  s(A)  p(t-X) 

j 

0 

1  T 

T 

,  If 

f  -1 

|  dt  S(t) 

j  dX  s(X)  sin 

P  (t-X) 

V  0 

J 

0 

(21) 


where  r(t)  -  p  ^(i)  .  1  1,  2  ...  M,  is  the  normalized  autocorrelation 

function  of  the  noise  ac  each  hydrophone. 

It  Is  a  simple  natter  to  show  that  Equation  (21)  can  never  exceed 

2 

unity.  For  greacer  ease  in  manipulation,  consider  the  quantity  1/R 
and  expand  the  inverse  sine  into  a  pcwer  series  (convergent  for  all  values 
of  t  and  *  since  j (  ( t  — A ) | <  1)  . 


1 


dt  s  ( t ) 


0 


T 

j  d‘  s(X)  [p(t-X)+l/2*l/3  p3(t-X)+l/2*3/4*l/5  p5(t-X)+...  ] 

_0_ _ _ 

ro  oj 

j  dt  s ( t )  J  dX  s ( X)  pft-X) 

0  0 


5,.  .. 


T  T  T  T 

dt  s(t)  J  dX  s(X)  o 3 ( t “ X )  j  dt  s(t)  J  dX  s(X)  pJ(t-'.) 

_  ,  +  ,  ,,  j) _ 0 _ ,  3_  _0 _ 0 _ . 

1  ‘  T  T  +  AO  T T  K" 


I 


dt  s(t)  /  d.X  s(X)  p(t-X) 


dt  s(t)  /  dX  s(X)  p(t-X) 


0 


0 


(22) 


:•  i  r.g  a  correlation  function,  is  positive  definite.  Hence,  the 


A  - 1  1 


denominator  in  each  term  is  positive,  regardless  o  the  form  of  s(t)  . 
Furthermore,  p(r)  Is  Fourier  inverse  of  a  non-negative  spectral  function 
G(w)  .  It  follows  that  p"(t)  is  the  Fourier  Inverse  of  the  n-fold  convo¬ 
lution  of  G(w)  wi th  itself.  This  convolution  is  clearly  non-negative, 
so  that  p n ( x )  is  positive  definite,  hence,  all  the  numerators  in  tqua- 
tion  (22)  are  non- uu  pat.  i  ve  .  f,u'.  ill  terms  of  the  equation  are  non-negative 
and  one  concludes 

-1,  >  1 

or  K2  ^  1  (23) 

In  order  to  set  a  lower  bound  on  R  it  is  necessary  to  make  subsi¬ 
diary  assumptions.  In  die  absence  of  any  restrictions  on  signal  and  noise 
properties  one  might  postulate  a  signal  confined  to  one  frequency  band  and 
n  noise  confined  to  a  different  disjoint,  frequency  band.  In  such  an  en¬ 
vironment  the  undipped  cut  tor  would  be  essentially  perfect,^  while,  the 
clipping  process  could  shift  appreciable  noise  power  into  the  signal  band, 
ihus  values  of  P.  arbit  r  u  :  .  v  .  .  to  ?.o  i  o  could  be  obtained  in  principle 
by  sufficiently  artificial  ■!  ices  of  sit’iul  atu:  noise  spectra.  Reasoning 
more  formally,  one  can  proceed  as  follows  to  establish  the  impossibility  of 
finding  a  general  Lower  bound  on  R  In  exr.-ss  of  zero: 

replace  the  inter  ml*  in  location  (??)  by  sums.  The  implied  sampling 
in  time  can  be  very  rapic ,  -•>  .at  :  b  .  jpp  r.  xicat  ion  is  arbitrarily  good. 


Limited  •  .  I  v  tv  >1.,-  <r  pi  I-  •  .  •  •  !•-,!  mu  ir.g  perfectly  disjoint 

spectra  with,  n  .  ul  -f  i  i.  i  u-  :  •  ion. 


.  tyf.i  ii  tens  in  Equation  (22)  can  then  be  written  in  the  font 


/  >  .  n 

n  v  V  r...  s.  s. 
_  iJ  i  i 

'  J 


S _ *_  5_  — _ 

{\  *  6 


— _(n_^2)  x  1 
—  (n  -  1)  n 


(ti  -  V  si  =  s(ti} 


|(a.  .  -  a,,)  +  a. 


i  s.  s, 
J-  i  J 


L  >_ 

i  .! 


a .  s  .  s  . 

ij  i  j 


7  j  (fij  ~  a,Msj 

\  N” 

/  /  .  a  ■  .  s .  s  , 

t  j  1J  1  J 


for  i  =  V. 


-  1  for  i  =  k 


U  all  other  i 


i  ,  -  ')  and  one  obtains  for  the  chosen  s. 

i  i  Li  i 


A 


+ 


V. 


(28) 


It  Id]  1  OU!',  I  rit 

'I  hi''  uprer 

time;-  t.,  nt'.u  !  .1. 

K 

upper  bound  .  hc|t;al  i  <  r. 

1 


1  bus  ,  for  any  f  i  :e  .. 
causi  K  t<  he  r  r"  1 : 

H' 

from  tin  case-.  past 
almost  rertair.lv  -  t 


wiie re  s  j  (  !  )  is 
.hapt )  ,.nd  :u 
bandwi  atli  1  -  ,  <  1  / 

u .  1  ‘  1  . :  .  ■ 


- ) 
!  ) 


(29) 


■.  ry  choc  sine  t  ho 


'  .  .  :  .  i  set  tier  .  Wilt)  A  at  this 

n 


y. ! 


I  in; 


(30) 


-  I  /  1  -  :< 


■  1  b  l  •  ;  tii.  ■  J  i  :,;nal  s(t)  which 


.  V  r  !  O  r  (  ■ 


ns  ,  its  very  dil ivrent 
1  t  ;  .  1.  :o|),ir  ,1  j.  IK.  1  won  1  d 


(31) 


t  .  ..  1 .  .  s  i  nnlnlnp  the  pulse 
,.-i  ■  r.odulat  Lon .  The 
■  is  11  romp  at  1  d  with 


Civ  i;  a  narrowband  signal .  it  appears  reasonable  to  assume  that  the 


•  re  bane  process  also.  For  if  the  noise  Is  primarily  rever- 
'.  irai  prepert  i<?<.  are  determined  by  the  signal,  while  in 
l"  nirnient  i  c-  1  ini  ted  environment  one  would  employ  filters  matched  to 
t b  nidwi  ■!•  ••  •  hv  si  gnal  ir.  order  r.o  improve  siennl- to-noise  ratio.  If 

:  no  Dopr  1  i  ! :  :  in  t  ;.o  target  signal  the  center  frequency  of  the 
noise  wi  11  •  w  1  . !..  o  .  (r  )  would  assume  the  form 

.  < •  )  =  p  j  ( !  )  cos  w^t  (32) 

i  io  bandwidth  of  i  ^(i)  is  small  compared  with  w^  .  Clearly  p ^ ( 0 )  ~  i 

tre  !  lor  all  t  .  Typical  forms  of  p  (r )  might  be 

1  2  1 
l  \  t  )  *■  Cil 

,  jO)  =>  «•  '  . ■•  ( •  )  =  e  both  of  which  have  the  computation- 

ill  . 11  ,  .  .city  .  j(t)  >  C  for  ail  t  .  This  property  will  be 

assumed  is  i  is  i  r..,..s :  ate  Lv  following  computations. 

Fs  it::  equations  ('.!),  (32)  and  t.25)  a  typical  term  of  Equation  (22) 
s an  now  in  wt  .  :  in  in  the  I nra 

1  '1 

,  i  .  (1)  cos  | i+t  ( t)  ]  cos  rwf)X+1;  (A)j  p^(t  )  cosnw^(L-X3 

Si  *’n  I  j 

It  s  (t)  s  (>)  cos[w()H  f(t)]  cos  [wpX+i})  (o)]  p^(t-X)  cos 

'(>  0 

(33) 


*Thi s  would  remain  e r rex i mat e Iv  true  for  targets  moving  sufficiently 
slowly  relative  i  t  Ik  :  .reiver  so  th.it  the  floppier  shift  is  small  compared 
with  the  sjen  ,1  :  atidwi  !lh . 


A-l. 


Since  n  Is  an  m..  l;  t-gtr  cos  w  (Z-X)  can  bt  expanded  as  follows 


cos  v  t.  ■  X !  •>  C 


The  constant -i  L 


part i cu  lat ,  t  r  ■ r 


" + - +  C  cos  nw  ( t-A ) 

nn  0 


^  3  A ) 


binomial  coefficients.  In 


In 


n 

u+  1 


(35) 


Substituting 
narrowband  ai...uinpt  t o: 
that  .mly  the  terr  t. 

1 1 

nif  Kandy  t  >  t.lu:  if  ■ 


< :  1 1 


,  .  :  (33)  and  invoking  the 

.  ft  1-.  .  v.s-u  !  eapue  lemma  to  argue 

■!  M4)  contributes  sig- 

uat  i  (  *31  becomes 


K  C, 
n  In 


;  -I 

i 

I  dt  u  ■ 

t  J 

0  0 


1 


>  I  \  -  • 


U  ,t+c(t.)j  cos  jw  A+4>(A)j  cos  w  (t-A) 


j  dt  j  (J/  V  ,  i  ■  /  ,  ( L -1  )  r  OS  j !Wnt+  i  (Lj  |  cos  f  V.^A+lfi  ( A  )  j  ros  W^t-*) 

0  ft 

(36) 


Straightforward  usi  !  ti  .  i 


■  .  yl  -  Ids 


W  !  »  -  t 


■■IS  W  (  t  -  »  ) 


y  ccc [g ( t ) -  ; - 

M 


cos  j  2WqC+«?  (t)+4>(*)] 


cos [ 2w  A+ * ( t.H-g (1 ) ]  (37) 

v 


On  ■■  mere  •  K 


ut  : 


tip  loved  to  eliminate  all 
1  !■(>'!  becomes 


r.  r  K  i . ,  0  0 

n  a  Ln  - - r; 

l  i 


|  dt  |  dA  s1(t)s1(A)p"(t-X)  cos  [<}>( t) -<J>(X)] 

0 _ 0 _  _  _  _ 

~T  T 

(  i 

j  lit  |  d  \  jU-\)  cos  |<?(l)-4.(A)] 


(38) 


0  «' 


All  factors  in  tin.  jute  rand  except  for  the  cosine  terms  are  non-negative. 

As  for  the  cosine  i  ■ rms  note  tiiat  f , ( t~A)  becomes  small  for  (t-A) 
larger  than  the  inrr'd.i.! an  time  ,J  the  noise  process.  For  reverberation 
this  correlation  citn  is  at  most  (stationary  sca’terers)  equal  to  the 
correlation  time  of  tne  signal,  which  in  turn  is  the  smaller  of  the  follow¬ 
ing  two  quantities:  1)  the  signal  duration,  2)  the  time  within  the  signal 
nulsu  during  which  the  modulation  4(t)  changes  by,  roughly,  one  radian, 
thus  the  cesiiv  terms  remain  positive  throughout  the  effective  region  of 
integration.1  rn  tin  ambient  noise  limited  case  the  noise  spectrum  would 

'•(.nornlly  1  c  shnpet,  by  a  a  no  1  tn.i  ti  ng,  filters  (or  possibly  transducer 


The  case  of  reverleratfon  from  stationary  scatterers  is  treated  with 

much  .  reatir  :  i :  ■  >  >  r  an:  or,.  :  a  !  i  in  Section  TV. 


,\-l  , 


character)  i  .  ai  a<  ;  ..e  have  a  u<i; idvi  dth  at.  least  equal  to  that  of  the 
signal.  Its  :  in  1  it  ion  title  would  therefore  be  no  longer  than  that  of 
t!  >>  1  gn...  1  ’  ti  ■  vt.  •  r.t  t  •  rirfir.s  valid. 

Under  these  cond i t i ens ,  the  integrands  in  both  numerator  and  denomi¬ 
nator  ot  iqu.it.  ti  .  ...  v  ..  r  the  effective  range  of  integration. 

Since  :  )  •  .  . 


A 


_1_ 

n+1 


(39) 


Hence  ,  t  run  1  j  vi . . .  : 

■  i  ,  .  +  2  '  ~-J-  ■  — .  .  .“Sy.U_L 

2  -  ■  ,  *■  -  '  y  n  - 1 )  ;  n+  J 

K 


(90) 


F roil,  the  wi  1 1 -Us. •  wu  .  :  1 1  \ 


( 2  *  9  '  <■  •  i  .  .  n+ . 

or.e  obtains  ,  upon  •  •  t 


(91)  ^ 


(92) 

.  ’  t  1  .  *  1  " 

n 

2 

It  is  now  a  s  imp  1  •_  ;  .at  i  >  r  to  ^'•und  j  /r,  •  roni  above  by  evaluating  the 

first  few  t"rrs  1  .  ;  n  '  ..  .  .  i:,-  .:  t  at  i .  a  ;  i  i  y  .ii’.d  using  Equation  (92) 

to  set  upp.  hour.  ■  i  r  Carrying  the  exact  computation 

t  rom  Lqua  1 1 1  (  i  .  t.i  ..  1 i »i:> 


l 


.  /-vlP. 


<  1.277 


1 


i 

R" 


(43)1 


Cor.'.L i uir.;;  Equations  (23)  .nd  ( b 3 )  one  tan  tl.oa- fore  rathe t  generally 
bound  the  clippinc  loss  for  noise  independent  from  hydrophone  to  hydro¬ 
phone  by 


0.89  •'  K  <  1 


(44) 


One  situation  in  which  the  assumptions  leading  to  Equation  (44)  are 
not  satisfied  is  that  of  an  ambient  noise  dominated  environment  contain¬ 
ing  a  strong  narrowband  noise  component,  here  the  effective  range  of 
integration  is  deienn  ned  by  the  correlation  time  i,  the  signal  component 
of  Equation  (38).  It  is  clear  from  intuitive  conside. ations  that 
'•os  j  *  <  t  )~4  0  >  ]  does  not  change  sign  over  this  interval.  More  specifically, 
consider  a  signal  of  the  form 


2 

’  1  N  2 

s( t)  =  e  cos (w^t+c,  t  ) 


(43) 


t  ion 


hubst  i  luti  on  lot  i, nation  (18)  yields,  after  an  elementary  cotnput.i 

I 


1 


T  2 


p"(x) 


A  K 
a  n  In 


?  > 

K  V\  2 

c  -  x 


(46)" 


Jx  e 


TV  V 


0  J  (x) 


i  2 

The  relation  1  +  1/9  4  i/25  +  1  /49  +  -  •  a  / 5  has  been  used  in 

si. taming  the  infinite  t erics  above  n  11  . 

‘The  limits  were  i  ntended  t  rota  (M  l  |  to  (-’,'•)  because  in  practice 
the  period  ol  inlet- rut  •  r.  would  undt  ubtedlv  cover  tfip  effective  duration  of 
the  pulse. 


A-  I : 


Since  both  integrands  of  Equation  (46)  are  non-negative  and  p^(x;  p^(x)  , 

Equation  (39)  land  hence  Equation  (44) J  rem-ins  valid. 

Thus  it  appeals  that  the  onlv  practically  important  exception  to 
Equation  (44)  [for  noise  independent  from  hydrophone  to  hydrophone] 
occurs  when  a  rapidly  moving  target  is  being  detected  in  a  reverberation 
limited  environment,  here  the  returned  signal  might  be  of  the  form 

sit)  =  s  (t)  cos[(wQ  +  WQ)  t  +  <J>  (t)]  (47) 

where  w  is  the  ooppler  sh.ift  caused  by  the  moving  target.'*'  Following 
the  same  procedure  as  in  Equations  (33)  to  (38)  one  arrives  at  the  fol¬ 
lowing  equivalent,  of  Equation  (38) 


A  K  C  ... 
ri  a  In  1 


T  T 

j  dt  ^  1)  Sj(t)  s  ^  ( A)  p^(t-A)  cos  [w^(t-A)  +  f(t)  -  (J>(A)] 

0  0 


(4b) 


J  dt  |  d As 
0  0 


^(t)  S^A)  c  ^  ( t  -  A )  cos  jw  ( t-A)  +  4>(  t. )  -  ( A )  ] 


As  a  specific  example,  consider 


,  .  JT  .  ,  K  2 

^(0  =  e  ,  <t(t)  =••  -  t 


(49) 


a  linearly  frequency  moduL.itec  pulse. 

It  is  a  simple  matter  to  demonstrate  [see  Equation  (£6)  with 
i  •-  j ,  d_  =  0]  that  j-  (t)  (for  reverberation  from  stationary  scatterers] 


The  signal  i  .  assumes  to  be  sufficiently  narrowband  so  that  the  Donpler 
shift  may  be  rr\  u  .  d  as  constant  throughout  the  bond. 


A-120 


assumes  tt  torm 


P  x  (  T  )  =  C 


1  ,2  2  \  2 
7  2  +  '  ai  “ 
2°T  “  6“ 


(50) 


2  0 


.  -.2  2 
5 + 


T 


may  be  interpreted  as  the  bandwidth  of  the  signal  and 


hence  of  the  reverberation. 

Substituting  Lquations  ( L )  and  (50)  into  Equation  (A8)  and  extending 
the  ranges  of  integration  to  (-“,  “)  ,  one  obtains,  after  some  algebraic 
manipulation 


A  = 


E  C,  exp  \ 
n  In  r 


n-1  1 

“d  S+I  'Z 


2  j 

-  +  K  0 
2  T 


(51) 


If  the  signal  bandwidth  is  determined  primarily  by  the  frequency  modulation 

22  A  ,  . 

K  a,.  >>  ”2  and  Lquation  (51)  becomes 

°T 


A  s  K  C.  exp  <w  K2o  2 

n  n  In  1  '  11  n+ 1 


D  n+1  1  T 


(52) 


'onversely,  if  K  o^.2  <<  ,  [little  or  no  frequency  modulation] 


»  ,  n-1  2  2 

A  =  K  C  exp  (  -7-—-  o.,  w 

n  n  In  \  A(n+])  1  D 


(53) 


In  either  case  the  coefficient  A^  becomes  large  when  greatly 

exceeds  the  bandwidth  of  the  signal.  Under  the  same  conditions  the  factor 
K  will  therefore  become  very  much  smaller  than  unity.  It  may,  in  fact, 
come  arbitrarily  close  to  zero  if  the  Doppler  shift  is  large  enough  compared 


A  -  2  L 


with  the  signal  bandwidth.  This  is  quite  reasonable  from  a  physical  point 
of  view:  Under  the  postulated  conditions  the  signal  and  reverberation 
spectra  are  essentially  disjoint  so  that  the  unclipped  detector  operates 
in  an  environment  that  is  almost  noise-free  in  the  signal  band.  Clipping, 
on  the  other  hand,  shifts  some  of  the  reverberation  power  into  the  signal 
band  and  therefore  sharply  degrades  detector  performance. 


A- 2  2 


IV.  Noise  Dependent  from  Hydrophone  to  hydrophone 


When  the  noise  field  exhibits  significant  coheience  from  hydrophone 
to  hydrophone  It  appears  to  be  difficult  to  obtain  results  of  the  same 
generality  as  in  the  previous  section.  Thus  even  in  the  absence  of  Doppler 
shifts  and  with  constraints  or  signal  and  noise,  such  as  those  specified  by 
tquations  (31)  and  (32),  one  can  readily  specify  realistic  spatial  covariances 
for  the  noise  which  result  in  values  of  P,  lower  than  0.89  [hquatlon  (44)] 
by  modest  amounts.  An  example  of  this  type  is  worked  out  in  Appendix  A, 
where  an  R  of  0.74  is  shown  to  be  attainable,  even  for  isotropic  ambient 
noise.  With  sufficient  ingenuity  one  suspects,  one  could  devise  noise 
models  yielding  even  lower  values  of  R  .  However,  even  the  simple  example 
of  Appendix  A  requires  fairly  special  assumptions  and  one  feels  that  the 
search  for  more  extreme,  cases  would  lead  to  more  and  more  artificial 
assumptions.  It  appears  more  rewarding,  therefore,  to  abandon  the  search 
for  extremes  and  turn  to  the  question  whether  serious  clipping  loss  is  likely 
to  occur  in  cases  commonLv  encountered  In  practice. 

Some  qualitative  insight  into  this  question  may  be  gained  by  restating 
Lquation  (20)  i.n  the  frequency  domain.  Defining 
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one  obtains  the  Fourier  inverses 


(54) 
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Substitution  of  Equations  (56)  and  (57)  into  Equation  (20)  yields 
(after  a  few  steps  of  computation) 


(58) 


where 
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(59) 


Since  the  observation  interval  (0,T) 

signal  pulse,  S(w)  is  in  effect  the 
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The  expression 
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G i  1  (w)  e 


would  generally  cover  the  entire 
Fourier  transform  of  the  signal. 

jw(i ,-ij) 

J  '  in  the  nunierator  of 


i=l  j  =  l 


Equation 
at  point 


(58) 

y. 


is  nothin);  ocher 
in  Figure  1  with 


than  the  normalized  noise  power  spectrum 

H  K  jw(t  -t.) 

the  clippers  removed.  y  Py  (w)e  J 

1=1  J=i 


ls  the  normalized  noise  spectrum  at  the  same  point  in  the  presence  of 
clipping.  Thus  the  numerator  (denominator)  integral  in  Equation  (5d) 
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represents  the  power  output  uf  a  filter  matched  to  the  signal  whose  inputs 
is  x  in  Liie  unclipped  (clipped)  instrumentation.  Since  clipping  tends  to 
spreau  the  spectrum,  one  wouic  ex  at  a  smaller  percentage  of  the  power  to 
fall  within  the  filter  band  in  the  clipped  than  in  the  unclipped  case. 

Hence  R  should  generally  tend  to  exceeu  /  2/n  =  0.8  .  Important  ex¬ 
ceptions  to  this  rule  would  b>  tApectec:  in  two  cases; 

1)  The  unclipped  noise  spectrum  is  centered  at  a  frequency  quite 

2 

different  from  tin  center  frequency  of  |S(w)|  .  This  would  be  the 

case  when  the  target  return  is  subject  to  a  strong  Doppler  shift. 
Reverberation  noise  is  not  subject  to  this  shift  and  it  is  only  throne! 
the  clipping  operation  that  a  significant  amount  or  noise  power  is 
transferred  into  the  signal  band. 

2)  Strong  negative  correlation  between  closely  adjacent  x^  In 
Figure  1  causes  the  terms  i  i  j  in  Equation  (58)  to  subtract  sub¬ 
stantially  from  the  power  input  into  the  filter  matched  to  the  signal. 
This  effect  can  result  in  values  of  R  smaller  than  /  2/tt  only  if 
clipping  reduces  die  negative  correlation  so  that  the  effect  is  less 
pronounced  in  the  denominator  than  in  the  numerator  of  Equation  (58). 
To  see  when  this  might  be  the  case,  consider  the  inverse  sine  in 
Equation  (20)  expanded  into  a  power  series,  as  in  III.  The  equivalent 
of  Equation  (22)  is  now 
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Unless  the  peak  negative  value  of  p 

for  n  significant  number  of  i  4  j  , 
denominator  does  not  take  place.  On 


Jj 


.  (t)  is  fairly  close  to  unity 


the  required  subtraction  in  the 


the  other  hand,  if  p„(t) 


comes  very  dost  to  (-1) 


Is  not  too  far  from  (-1)  and 


a  similar  subtractici  takes  place  in  the  numerator.  Thus  one  looks 
for  maximum  clipping  loss  in  cases  where  the  negative  correlation  be¬ 
tween  closely  adjacent  phone's  is  strong,  but  not  strong  enough  so 

a 

that  p^’(i)||ir,x  ha:  a  magnitude  comparable  to  unity.  Since  these 

conditions  on  the  ,  . .  are  quite  restrictive  one  is  not  surprised  to 

find  only  modest  Qi creases  in  R  for  even  rather  carefully  con¬ 
structed  i xrmp 1 es ,  such  as  the  situation  analyzed  in  Appendix  A. 


dote  that  the  rather  rapidly  converging  sequence  of  coefficients  in 
Lquation  (60)  demands  integral  ratios  of  at  least  the  order  of  5  before 
,/l\2  begins  to  increase  very  substantially. 
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To  complete  the  discussion  the  clipping  loss  will  now  be  computed 
for  a  fairly  general  case  of  operation  in  a  reverberation  limited  environ¬ 
ment.  The  most  important  restriction  is  the  assumption  of  negiiblu  Doppler 
shift,  both  in  the  target  return  and  in  the  reverberation.1 

The  key  step  is  the  computation  of  the  cross-correlation  of  the  revei- 
beration  at  the  i*1'1  and  j  hydrophone.  If  the  signal  assumes  the  narrow 
band  form  of  Equation  (31)  the  reverberation  observed  at  the  i*”''  hydrophone 
is 


V  (t)  =  — ■ -  s  (t  c  )  coslw  (t-t  )  +  4>(t-c  )]  (bl; 

i  — -  2  1  £  1  0  2  £  J 

£ 

t  is  the  travel  time  of  sound  from  the  origin  of  coordinates  (nominal 
center  of  the  source)  to  the  2^  scatterer  and  back  to  the  i^'  hydro¬ 
phone.  me; sures  t lie  amplitude  of  the  signal  reflected  by  the  £t 

scatterer.  It  includes  effects  of  the  transmitter  beam  pattern  as  well 

2 

as  those  of  scatterer  cross-section.  Similarly,  the  reverberation  at 
the  j t1'  hydrophone  is 

v.(c)  /  — ~r  s  (t-’l  )  cos'w  (t-T  )  +  4>(  t--T  )  j 

j  ^  i  f  ‘-o  i.  .£  4 

i 

1  is  the  sound  travel  time  from  the  origin  to  the  j1"*1  hydrophone  via 
the  2  "  scatterer.  hence  the  desired  cross-correlation  assumes  the  f  r 
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Lxpressinp  the  cos(  )  cos(  )  product  in  terms  of  sum  and  difference 
freouen  ies  one  can  invoke  the  Piemann-Lebesgue  lemma  to  eliminate  all 
but  the  difference  terms  of  form  l  =  m  .  Heiwe 
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The  next  step  is  to  express  T  in  terms  of  t  .  Consider  the 

l  i 

spherical  coordinate  system  shown  in  Figure  2.  By  arbitrary  convention 

.  th 

l  scatterer 


The  symbol  E{  }  denotes  the  expectation  of  the  bracketed  quantity. 
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the  origin  is  placed  at  the  i1^1  hydrophone,  u , .  is  the  distance  between 
phones  i  and  j  .  A  simple  trigonometric  computation  now  yields  the 

distance  R,  ot  t  lie  tt1*1  scitierer  !  r .  >u  no  j  in  terms  of  r  ,  the 

distance  of  the  l*"*'  scatteror  from  phone  1  . 
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(64) 


In  practici  -- -  <<.  1  for  all  stutterers  sufficiently  close  to  the 
1  f 

target  to  receive  some  illumination  simultaneously  with  the  target.  Using 
this  approximation  and  dividing  both  sides  of  liquation  ((■>>■'  by  the  velocity 
m’  sound  cue  obtains 
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For  greater  ease  in  subsequent  manipulation  we  introduce  the  riota- 
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iiie  joint  prcb.il)  i  1  :  t  v  ci.uv  civ  <>t  i  .  ■:  and  u  has  been  calculated 


"i 


in  Report  No.  27,  For  volume  reverberation  from  scatterers  independently 
and  uniformly  distributed  over  a  large  volume  V  the  result  is 
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The.  coefficient  a^  is  now  decomposed  into  its  two  primary 


components 
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(69) 


b is  proportional  to  the  scattering  cross-section  while  g(0,  4>)  is 
the  transmitter  pattern,3  centered  at  (0  ,  (j>^)  . 

With  the  introduction  of  Equations  (68)  and  (69)  and  the  change  of 
variable 
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Lquation  (67)  becomes 
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The  use  of  a  frequ>  ncy  independent  pattern  function  Implies  that  the 
signal  is  sufficiently  narrow-band  to  have  its  directional  properties 
described  by  a  single  frequency  pattern  function. 

2 

The  bar  indicates  an  averaging  operation  over  the  number  of  illuminated 
scatterers . 
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{'hr  slowly  varying  amplitude  fact.  *  b ,  /  L  -  1  —  j 
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and  negative  deviations  are  allowed)  and  a  maximum  spacing  between 
hydrophones  of  20  ft.,  one  finds  that 


max 


0.4n 


(73) 


Approximation  2)  has  certainly  become  questionable  for  these 

parameters.  However,  in  any  practical  array  a  very  small  percentage  of 

the  total  hydrophone  pairs  have  spacings  close  to  the  maximum  array 

dimension.  Furthermore,  one  tends  to  use  arrays  in  such  a  manner  that 

few,  if  any.  hydrophone  pairs  assume  an  endfirc  alignment  relative  to  the 

d .  . 

ij 

target.  Hence,  a  will  generally  be  well  below  unity  and  a  — ~  in  a 
2C  ft.  array  would  be  substantially  below  0.004  sec  for  all,  or  almost 
all,  pairs.  Hence,  approximation  2)  appears  not  unreasonable  for  arrays 
of  this  general  sit*.  and  bandwidths  up  to  the  order  of  100  cps 
With  approximations  1)  and  2)  Equation  (71)  becomes 
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Assuming,  for  the  sake  of  computational  simplicity,  that 
and  ij (  )  are  both  even,  i.e.. 


By  introducing  an  assumption  of  narrow  beam  patterns  one  can,  in 

j  „  Vj 

fac.,  considerably  weaken  the  inequality  i  _  1)  „  — i  q  implied 

by  2)  .  The  appropriat.  weaker  inequality  i.'  worked  out  in  Appendix  B. 
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(75) 


and 

4)(x)  -■  4, ( —  x)  ev.n  phase  modulation  or  odd  frequency  (76) 

modulation,  such  as  linear  FM 

one  can  readily  reduce  Equation  (741  to  the  form 
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I  In.  double  integral  in  Equation  (77)  depends  only  on  transmitter  and 
receiver  geometry,  while  p(i)  depends  only  on  the  waveshape  of  the 
transmitted  signal.  Tin  complete  separation  of  these  two  effects  is  the 
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where 
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With  the  changes  of  variables  0^-0^  =  u>  =  V  tYlG  doukie  i-ntegral 
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According  to  Equation  (20)  we  are  ultimately  interested  in  the  value 
of  the  autocorrelation  function  not  at  x  ,  but  at  t  -  X  -  +  t.  • 

However,  with  the  array  steered  on  target  we  obtain  from  Equations  (65)  and 

(66) 
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Hence,  from  Equations  (77),  (83)  and  (84),  using  the  postulated  symmetry 
of  the  pattern  function  about  (0^,  4>q) 
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Thus  G(w,  z)  is  the  Fourier  transform  of  the  pattern  function  g(u,  v) 
To  normalize  the  crosscovrelatioo  function  we  need  only  recognize 
from  Equation  (85)  that 
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Equation  (88)  must  now  be  substituted  into  Equation  ^bJ).  Designating  the 
general  term  of  Equation  (60)  by  A  as  before  and  using  an  obvious 
generalization  of  the  steps  leading  to  Equation  (38)  one  finds 
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g(u,  v)  describes  the  spatial  distribution  of  radiated  power  and  is 
therefore  non-negative.  It  follows  from  Equation  (86)  that 

G(u,  v)  _<  G(0,  0)  for  all  u,  v  (90) 

Furthermore,  it  is  shown  in  Appendix  C  that  for  narrow  beam  patterns 

G (u ,  v)  >  0  (91) 

The  ratio  of  the  double  sums  Equation  (89)  is  therefore  no  larger  than 
unity  and  one  obtains 
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P (t-A) 
.  p(0) 


A  <  K  C, 
n  —  n  In 


X)  pa  , 

dt  s  1  ( t )  j  dA  8X(X)  cos  [4>(t)-ij(X)  J 


(92) 


Using  Equations  (75)  and  (76)  it  is  a  simple  matter  to  show  further  that 

a>  oo 

ft,  n i  fix  i  p(t-x)ln 

J  dc  si(t)  0  J  dx  si(x)  L  TloTJ 


A  <  K  C, 
n  —  n  In 


(93) 


CO  CO 

f  dt  S](t)  Cj$(t)  f  dx  Sjd)  *TJ*(X) 


J  1  J 

—  rr.  —on 

Thi  fact  that  both  double  integrals  are  in  form  of  convolutions 
suggests  the  use  of  Fourier  transforms.  Define 


S (w) 

<■  I 


OO 

r 


si(t) 


cJ  <i(t) 


-  |Wt 
e 


dt 


(94) 


and 


<r> 

P(w)  =  j  p(t)  e  ^Wt:  dt 

—  On 


(95) 


Thus  S^(w)  is  the  Fourier  transform  of  the  low  frequency  signal,  i.e., 

the  signal  after  a  downward  shift  by  w  In  terms  of  Equations  ( 9 4 1 

and  (95)  A  can  now  be  written  as  follow:-; 
n 


dw  | S  , (w) | “  tP(w)  *  P(w)  *  ...  *  P(w)} 

2,1 


A  <  K  C.  Ip (0)  1 
n  —  n  lntr 


'r(n-l) 


(96) 


J  dw  | 3^ f (w) | 2  P(w) 


Here  A  *  B  denotes  the  convolution  of  A  with  B  . 

Consider  next  the  relation  between  P(w)  and  S^^(w)  .  Using  the 
postulated  symmetry  of  s^  and  $  one  obtains  from  Equation  (78) 


p(T) 


•x 

f  ,  ,  T,  .  .  \\  j<J>(x  +  ~)  —  j  d>  (x  -  t!  ) 

I  dx  s1(x  -  -j)  s^fx  t-  -)  eJ  2  t  2 

—  CO 


(97) 


The  change  of  variable  x  +  ~  =  y  leads  to 

Ot) 

p(.)  -  j  dy  spy)  s)t(y)  spy  -  ,)  (98) 

—  00 

which  is  a  convolution  of  s^(y)  ^  with  Sj(y)  e  .  It 

follows  that 


P(w)  =  |Sfcf (w) |2  (99) 

Thus,  P(w)  is  real.  Now  substituting  Equation  (99)  into 
Equation  (96)  and  using  Farseval' 


th.  orec, 


J  dw  P(w)  {P(w)  *  P(w)  *  ...  *  P(w) 

A  <  K  C  [p(0)]"(n-1)  — - 

n  -  n  lnir  1 


dw  P(w)  ■  P(w) 


p(T) 

p  (o) 


1  n+1 


=  K  C 


n  In  " 


dT 


P(T) 

p(0) 


(100) 


Since  n  assumes  only  odd  values,  the  integrands  of  both  numerator 
and  denominator  are  non-negative.  Furthermore,  from  Equation  (99) 


I P (T )  1=2 j-  J  dw|s  .f  (w) 


2  <JWT 


1 

2v 


dw|s_  f  (w)  |4  =  p  ( 0 ) 


(101) 


hence,  the  ratio  of  integrals  in  Equation  (100)  has  an  upper  bound  of 
unity  and 

A  *  K  C.  (102) 

n  -•  n  In 

but  this  is  identical  with  Equation  (39)  so  that  one  obtains 
immediately  from  Equation  (44) 

R  >  0.89  (103) 

Thus,  at  least  in.  the  .absence  of  Doppler  shifts,  clipping  losses 
in  a  reverberation  limited  environment  are  quite  small  for  a  very  general 
class  of  signals  and  arrays. 


\-:is 


V.  Range  .stlmatlon 


An  estimate  of  target  range  can  be  obtained  by  regarding  the  correlator 
output  as  a  function  of  replica  delay  (t)  and  establishing  the  location11 
of  this  function  on  the  t  axis.  Details  of  the  required  instrumentation 
depend  on  the  precise  definition  of  the  tern  "location",  b 't  if  the  signal 
correlation  function  is  sufficiently  concentrated  in  t  to  permit  useful 
range  estimates  one  would  expect  any  reasonable  measure  of  "location"  to 
lead  to  comparable  results.  One  such  measure  is  obtained  by  the  arrangement 
shown  in  Figure  3.  x(t)  is  the  output  of  the  bcamformcr  as  in  Figure  1. 
s(t-t1) 


x(t) 


1 

----- (x) 
multiplier 


multiplier 


integrator 


-©- 

f 


s(t-t„) 


Figure  3 


It  is  cross-correlated  with  two  replicas  of  the  signal,  delayed  by  t 
and  t^  seconds  respectively.  The  resulting  short  time  correlation 
functions  c.  "  subtracted  to  yield  the  final  output  z(t)  .  If  the  target 
delay  (tg)  is  given  by 


t 


0 


(104) 


the  expected  value  of  z  is  zero.  Deviations  of  z  from  zero  indicate 


A-.  0 


values  of  target  delay  other  than  that  given  in  Equation  (104).  The 
measurement  error  of  such  an  instrumentation  has  been  discussed  in  Report 
No.  29  (Section  I).  If  the  true  target  delay  is  tg  then  the  rms  measure¬ 
ment  error  is 


0-05) 1 


This  figure  of  merit  must  now  be  calculated  for  instrumentations  with 
and  without  clipping.  In  the  absence  of  clipping  the  equivalent  of 
Equation  (2)  is 

x(t)  =  M  s ( t  —  t  )  +  )  n. (t-r .)  (106)2 

o  L _ i  i  i 

i=i 


Hence , 


M 

y1(t)  =  M  s(t-tg)  s  ( t — t  ^ )  t-  N  ni  (t-xi)  s(t-t1)  (107) 

i=l 


For  sufficiently  high  signal-to-noise  ratio  to  make  meaningful  range 
measurement  possible. 

2 

The  target  delay  t^  was  emitted  in  Eouation  (1) .  Since  range  was 
assumed  to  be  known  in  the  detection  study  all  delays  could  be  measured 
relative  to  Uq  . 


A-40 


and 


y^it)  =  M  s(t-0  s(t-t.)  +  n^i;-^)  s(t- 


LO.i. 


id 


Since  the  average  value  of  the  noise  is  zttv 


M 


T  j  dt  s(t-tQ)  [s(t-t1> 
0 


s(t-t^) J 


(109) 


Similarly  in  the  presence  of  clipping  one  obtains  from  a  computation 
parallel  to  Equations  (2)  -  (10)  [low  input  signal-to-noise  ratio] 


2  M 


T  VN 


-  j  dt  s(t-tg)  l s ( t-t 1)  -  s(t-t2)] 


UiO) 


hence,  by  a  trivial  computation 


3z_ 

"3tC 

dz 

3t, 


clipped 


undipped 


2 

it  N 


In  the  absence  of  clipping  the  mean  sqiare  value  of  z  is 


(111) 


T  H  T  M 

|  j  dt  s(t-t1)  )  n± ( c jl)  "  I'  j  dt  s(t-t2)  ^  Oj  (t-i^) 

0  i=l  0  i=l 

T  J 

\  J  dt  J  dx [s (t-t j )s( \ -t^)  2s (t-c j ) - { \-t2)+s(t-t2) s (X-t2)  ]  x 
1  0  0 


\  \ 
/_  /_, 

id  jd 


pd(t'A"Ti+Tj) 


(112) 


The  contribution  <  :  . !s 


ionr  t  z  has  been  Lgnc 


under  the  assumption  of  Low  input  s igna» -to -noise  ratio. 


\-  i  : 


Similarly  in  the  presence  of  clipping  see  Equations  (11)  -  (13) 


z2  =  — 


dt  j  di [s(t-tJ)s(X-t^)-2s(t-t1)s(X--t2)+s(t-t2)s(X-t2) 


0  0 


x  — )  )  sin  I  p .  .  (t-X-x  ,  +  t  . )  ] 

it  4 — /  l. — ■  ~  ij  i  j 

1=1  j-1 


(113) 


From  Equations  (105)  and  (111)  -  (113)  the  clipping  loss  is 


''O  clipped 


0 1  undipped 


T  T 


J  dt 

)  _0 _ _0 


dX  |  s  f '  )s(X-t  )-2s(t-f1)s(X-t2>  i-s(t-t0)s(,X-t2) 


dt  J  dx[s(t-t1)s(X-t1)-23(t-t1)6(A-t2)+s(t-t2)s(X-t2)] 
0  0 


_M  _M 

L  LDij(t-'-Ti+Tj) 

i=i  j-i 


M  M 

T  ^  •  -1 


)_  sin  [pjjlt-X-T.+T.)] 


(114) 


i=i  j-i  ; 

The  similarity  with  Equation  (20)  is  obvious.  One  can  clearly  carry 
through  many  of  the  gcn..ral  arguments  of  Section  III  and  obtain  similar 
results.  Here  we  shall  concern  ourselves  only  with  the  case  of  reverbera¬ 
tion  In  the  absence  of  Doppler  shiftjt  retted  in  Section  IV,  Equations 
(61)  -  (93)1.  Once  again  we  work  with  the  general  narrowband  signal 


s(t)  =  s^(t)[cos  w  t  +  * t )  j 


(115) 


A  -12 


With  the  same  restrictions  on  array  dimensions  as  in  Section  IV 
(array  diameter  small  compared  with  wavelength  of  maximum  frequency 

l- 

deviation  from  w^j  ,  one  obtains  from  Equations  (60)  and  (88) 


s(t-t^)s(A-t0)  -2s(t- t1)s(X-t2)+s(t-t2)s(X-t^)  cos  wQ(t-A) 


where  is  defined  by  Equation  (25),  p(t)  by  Equation  (78),  and  A^ 

t  h  2 

is  the  n  term  of  the  expansion  of  1/R  .  The  limits  of  integration 

have  been  extended  to  (-«,  °°)  on  the  assumption  that  the  integration 
time  (0;  T)  at  least  covers  the  duration  of  the  two  replicas.  As  in 
Section  IV,  the  ratio  of  the  double  suns  has  an  upp^r  bound  of  unity. 


A— hi 


Only  the  '■era  in  s(t-t^)  s(t-t2)  differs  from  the  forms  treated 


previously.  One  can  express  the  constraint  of  Equation  (104)  by 


t2  -  tQ  +  A 


C1  ’  tQ  -  A 


(118) 

(119) 


Substituting  Equations  (115),  (118)  and  (119)  Into  (117),  one  obtains 
after  some  algebraic  manipulation  (invoking  the  Riemann-Lebesgue  lemma) 


00  00 


dt  I  dA{s^(t) s^(A) cos  [$(t)-$( A) ]  -s^(t+A)  s ^  (A -A) 


—  CO  —CO 

A  <  KC,  - 

n  -  n  in  oo 


dt  dx{ s^( t) s^ (A) cos  [<f)(t)-<j>(X)  j-s^t+Ajs.  (X-A) 


—  oo  —  rn 


cos  [2w  A+4> ( t+ a)  - cj> (  X— A )  ]  ) 


p(t-X) 

jlQ}- 


cos  j  2w_A+<J>(t+A)  — q> (  X— A)  j  }  — 

°  p(0) 


(170) 


The  signal  correlation  functica  is  monotone  only  over  intervals  of 
the  ord^r  of  half  a  carrier  cycl<_.  Hence .  the  separation  (2^)  between 
the  two  replica  delays  cannot  exceed  tt/w^  if  the  output  from  the  device 
of  Figure  3  is  to  have  an  unambiguous  interpretation.^  For  time  increments 
of  the  order  of  u/w^  the  relatively  slowly  varying  functions  s^(t)  and 
( t )  do  not  change  significantly.  Hence, 


The  practically  more  interesting  situation  in  which  only  the 
of  the  correlation  function  is  used  in  ranging  is  discussed  on  p. 


envelope 

42. 


A-M 


CO  CO 


dtj  dXs^(t+A)  s^(  A)  C06  [2WgAf  i}i(  t+ A)  -$(  X-A)  ] 

)  —  CO 

)  oo 

dt  I  dxs^(t)s^(x)cos  [2Wq Af  4>(t )-(}>( A)  ] 


p  (t-X) 
p(0) 


"CO  —co 


CO  CO 


cos2WqA  I  dt  |  dAs^(t' s^(\)cos  [$(t)-4*(  A)  ] 


p(t-A) 

p(0) 


in 


—  CO  —  05 


Or  co 


-  sin2WgA  I  dt  d  \b^( t)  (  X)  sin  [i|»(t) -  $(  A)  ] 


EiLrl) 

p(0) 


—  CC  -  CO 


The  change  of  variable 


t  -  X  °  x 
t  +  X  **  y 


yields 


CO  CD 


j  dt  /  dAs1(t)s1(  A)sin  [t>(t)-<j)(  A)  ] 


p(0) 


-  CO  —  oo 


J 


p(x) 

p(0) 


’  dy s J  (*£*>  9 A  ( yf-)  s in  f^)  -  *(*f  >] 


(121) 


(122) 


(123) 


y+x  y —X 

and  (j,  are  even  functions  by  assumption.  Therefore  'f1  {~~j~ )  —4*  rap ) 
is  odd  in  y  .  It  follows  that  the  integrand  oF  Equation  (123)  is  odd 
in  y  so  that  the  value  of  the  integral  is  zero.  Using  this  result  in 
Equation  (121)  and  substituting  in  Equation  (120)  one  obtains  finally 


CO  OO 


dt  /  jxs1(t)s1(x)cos  [o(t)-<t,(A)  ] 


p(t-*) 

p(0) 


A  <  K  C 
n  —  n  In 


“  tx>  —  CO 


(124) 


CO  oo 


J ■“  I 

—  CO  —  00 


d As. ( t)  s.  (  A) cos  |  i(  t )  -v(  A)  I 

11  p(Oi 


A— I  •> 


Hut  this  i  i»  identical  with  Lquat  on  (92)  no  that  Kquotion  (103)  remains 
t  rue  : 

K  ^  0.89  (123) 

Thu  d  I  ficuss  1  on  |  um  t  concluded  lr.  unrealistic  In  one  respect.  The 
postulated  Instrumentation  ur.cn  the  carrier  frequency  In  ranging ,  thn*. 
obtaining  In  effect  extreme  range  accuracy  (to  .a  small  fraction  of  a 
carrier  wavelength)  at  the  expctific  of  ambiguity  over  multiple.'!  ol  the 
carrier  wavelength.  In  practice  one  cannot  tolerate  uuch  ambiguity.  One 
would  therefore  almost  certainly  Ignore  carrier  frequency  elle<-tn  and  neek 
to  locate  the  envelope  of  the  signal  correlation  function  on  the  i  axis. 

Tlu  formal  analysis  of  an  appropriate  1  util  rumen  tat  ton  In  fairly  cumber  mime , 
but  uuch  ol  the  desired  Insight  Into  the  question  ol  clipping,  loan  can  he 
obtained  (  rom  Lite  lol  lowing,  line  of  reasoning. 

A  typical  signal  autocorrelation  function  It;  sketched  In  Figure  9. 

H  (i) 

it 

A  '  ‘  A 


i 


j 

I 


i 


[ 


Figure  A 

sit  Instrument  at  ion  of  the  type  of  Figun  t  <h  .igned  to  track  the  i  hvi  lone 
would  have  lit  compare  the  amplitude  ol  the  quas  -sinusoidal  one  I  I  I  a t  I ons 
n.t  a  distance  A  f  rom  tlu  origin.  This  clearly  lends  to  lower  ;n  to.  1 1  1  v  1  t  I  eft 
than  In  tin  pnvlons  compulation,  where  A  was  of  the  order  ol  a 
quarter  wavelength  ol  the  mrrlir  Inquest  v.  On  the  othei  hand.  It  Is  dear 


\-  H 


from  Equations  such  as  (109)  and  (110)  that  the  ratio  of  sensitivities  for 
the  clipped  and  unclipped  instrumentations  [Equation  (111)]  is  unaffected 
by  this  change.  Furthermore,  the  output  fluctuation  z"  for  the  envelooe 
observation  is  simply  the  output  fluctuation  of  Figure  3  with  the  delay 
adjusted  for  operation  at  the  peak  of  the  appropriate  carrier  cycle  (as 
suggested  in  Figure  A).  Thus,  Equation  (114)  and  hence.  Equation  (120)  is 
still  indicative  of  the  clipping  loss  if  A  assumes  the  appropriate  value. 
One  can  now  no  longer  make  the  approximation  in  the  second  lint  of  Equation 
(121)  and  must  write  instead 


S».  IX 


dt  dX  s,(t+A)  s  (X-A)  cos  [2w  A  +  <J>(t+A)  -  £(X-A)J 


PlT-rA) 

p(0) 


—  03  -  CO 


CO  OO 


:os  2wqA  dt  j  dx  s^t+A)  s^CX-A)  eos|<?(t+A)  -  d(X-A)] 


p(t-X  ) 

p(0)  J 


—  or  -co 


co  co 


f  f  - 

sin  2uqA  ,  dt  I  dx  s^t+A)  s^U-A)  sin[<t«(t+A)  -  p(X-A)J 


p(0) 


—  OO  — OO 


(126) 


The  change  of  variable  (122)  applied  to  the  last  term  of  Equation 
(126)  still  leaus  to  an  odd  y  function  so  that  this  integral  vanishes. 
The  equivalent  of  Equation  (124)  is  therefore 


1  TT 

The  zeros  of  R  (t)  occur  at  values  of  t  such  that  w_t  *  K— 
s  u  L 

K  odd.  However,  the  peaks  of  R^(f)  do  no-  necessarily  occur  midway 
between  the  zeros,  a  fact  which  causes  an  apparent  difficulty  in  the  precise- 
choice  of  A  .  Fortunately,  it  turns  out  that  changes  of  A  by  a  small 
'faction  of  a  carrier  cycle  do  not  ,af f  <•*  <-n,-  clipping  loss  computation. 


A—)  7 


on  cn 


f  [ 

1  dt  dA 

pJLtjiAl 

i  J 

p(0) 

l  S  tl)  8  ^  (A)coS  f<j>(t)  -(}■(  A)  | 


A  K  C, 
ri  —  n  In 


—  OO  —  to 


00  00 


dt  dX-^~ ~^{ai(t)a1(X)coo|i!>(t)-<>(A)  | 


—  on  _(*> 


-  coa2w^A.s  j(t+A)Sj  (  X — A )  c  os  |  <J>  ( t+ A)  —  (J>  ( A  -  A )  |  } 

-  cufi2w^As  (  t+A)y  ( A-A)  cob  |<J>(  t+A) -<!>(  A-A)  |  I 


(127) 


The  first  terms  in  numerator  and  denominator  are  identical  with  the 
Integrals  In  Equation  (02).  The  fiecond  terms  ran  be  reduced  to  an 
analogous  form  by  the  transformation 


x  ”  t  +  A 
y  “  t  -  A 


(1 28) 


Following  the  same  sequence  of  steps  as  In  hqu  tlonr.  (‘121  -  (100) 
mil  now  obtains 


d  i  | 


I'ijl  |  1  _  co.  /w  A  [o(.0|n  , 

1  1  Ul  rv*  L/n  \  I  ,./n\  ' 


p(0) 


0°  ][)(())  I  |.(0) 


I  •  K  C, 
u  u  111 


/  cl  r  < 

P.( ') 

p  ( 0 ) 

-  cos  2w^A 


llLiJ 

p(0) 


P  ( '  zJ-  ■  V 

p(0)'  ‘ 


( 1  '•’')) 


Hi  cause  of  the  mgativi  tiii.is  it  is  dlifiitill  to  draw  umclutdim;;  ol 
the  Hattie  generality  as  In  tlu.  case  of  d.Krtlou  jlquitions  (100)  (I0/)|. 

'Hie  practlcil  picture  becomes  i  liar,  however,  when  no  recalls  that 
p(i)  |  Equal  Ion  ('17)  |  t  ii  in  i  I  I  ect  tiu  lutoeoriel  t  ion  function  H  (i)  < '  f 

tin  signal.  Furlhe  ntiori  ,  A  nun  I  he  rhou«  n  in  ring/  of  :  vain,  a  win  1 1 
!•'  (i)  decays  raplllv.  II  tin  eiivi  lope  ol  It  (i)  Is  ;,u  f  f  1  c  1 1  n  t  I  y 


A  1  - 


concentrated  on  the  i  axis  to  permit  meaningful  range  raeasurcXn. nth 

in 


£iTi 

p(0) 


p (t-2 A) 


P(0) 


should 


p ( 2 A)  should  be  very  small.  Hence,  the  product 
be  small  for  all  t  and  the  second  terms  in  numerator  and  denominate r  of 
Equation  (129)  should  have  only  a  minor  effect  on  the  ratio  of  integrals. 
Thus,  one  expects  Equation  (102)  to  remain  true  with,  at  most,  minor 
modifications . 

As  an  example,  consider  a  signal  consisting  of  a  Gaussian  pulse  with 
linear  frequency  modulation 

2 


UT  K  2 

s(t)  =  e  cos(w^t  +  -  t^) 


A  straightforward  computation  yields 


P(t) 

p(0) 


o2  2 
fl  T 


where 


J.  2 


Thus,  f)  is  the  effective  bandwidth  of  the  transmitted  signal. 
Substituting  Equation  (131)  into  Equation  (129)  one  obtains 


A  <  K  c,  V ZZ7 
n  —  n  In  V  n+1 


1  -  cos  2WpA 


“  -2-  4  pY 

n+1  q  U 


1  -  cos  2w  A  c 

o 


2  2 

-  2  PA 


p(t)/p(0)  has  maximum  slope  at  t  =  1  — '  .  Hence,  this  value  of 

V 2 

delay  is  chosen  for  A  .  T’ticn 


(130) 


(131) 


(132) 


(133) 


A— 10 


The  last  fraction  in  Equation  (134)  has  a  maximum  value  when  the  cosine 
terms  have  arguments  equal  to  multiples  of  2r  and  when  n  00  .  There 
foie 


A  <  KC, 
n  -  n  In 


1.37  K  C 
n 


In 


n  assumes  only  odd  values.  For  n  =  3 


1.37  K  C 
n 


In 


0.966  R  C 


It  follows  that 

A  <  K  C,  for  all  n  3 
n  —  n  In 

Since  A^  =  1  0.89  remains  a  lower  bound  on  R  .  In  this  particular 

example  the  lower  bound  is  definitely  not  reached,  so  that  the  clipping 


loss  is  actually  even  smaller. 


VI,  Doppler  Estimation 

Radial  target  velocity  can  be  estimated  by  a  procedure  very  similar 
to  the  one  used  in  range  estimation.  Figure  5  shows  n  schematic  Doppler 
estimator  equivalent  to  the  range  estimator  of  Figure  3.  The  target 
delay  is  assumed  to  be  known  and  the  signal  is 


(134) 


(135) 


(13  j) 


(137) 
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s^(t)cos[w^t+4-(t)] 


1 - VS.’ - 1 

integrator 

j 

5. 

T 

rf  dt  y(t) 
0 

l  . 

x(t)  =  s1(t)cos  [(wQ+wD)t+«  (t)j 

-  yCO  j 

S.(t)cO£  j  w7  tJ  $  (t)  ] 


Figure  5 


L  ken  as  sufficiently  narrow-band  so  that  Doppler  shifts  in  the  envelope 
and  phase  modulation  nay  be  ignored. 


By  analogy  with  Equation  (105)  the  rrns  Doppler  error  is 


^  V"2 

“o+”d  "  ~7~ 


\J  f-w 

0  D 


wl+w2 


(138) 


A  computation  entirely  parallel  to  Equations  (106)  -  (111)  verifies  hat 


c*  Z 


clipped 


undipped 


Proceeding  as  it.  Faction  IV  one  obtains  in  place  of  Equation  (117) 


n  <  K 


'  P  I 
n  n 


I),  + 


P 

ri 
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(139) 
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where 


CD  CO 


*  dt  J  dXs^(t) s^(X) cos  [w^t+(fi(t)  Jcos  [w ^ A+  (> ( X ") ] 


P(t~X) 


in 


P(0) 


cos  w^(t-X) 


—  03  —CO 


(141) 


oo  oo 


»■  j  dt  dAs^(t)s^(A)cos  [w2t+cj)(t)  ]cos  [w^A+^U)] 


P(t-X) 


p(0) 


cos  w^t-X) 


—  oo  —CO 


(142) 


CO  00 


F  = 
n 


dt  f  dxs1(t)s1(A)cos  fw^t+^Ct)]  cos  [w2X+<*>(x)J 


P(t-l) 


p(0) 


cos  wQ(t-x) 


-or,  —co 


(143) 


Set 


W1  =  W0+WD  ”  A  W 


W2  =  W0+WD  +  A  w 


(144) 


Then  a  sinple  computation  yields 


00  CO 


D^+E^  =  dt  dxs^(t)s^(x)cosAw(t-x)cos [w^( t-x)+<|i  (t)-ij»(x)] 


P(t-l) 


L  P(0)  J 


— CC  —00 


(145) 


With  the  change  of  variable 


t  -  X  =  x 
t  +  X  =  y 


(146) 


one  obtains  (using  the  even  symmetry  of  (J5) 

CO  00 
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D  +E  =  ~  dx 
n  n  4  j 


p(x) 


P(0)J 


cosAwxcosv^x  dys^(^y^)s^(^^-)cos[<)(^y^)-<t>('^^)] 


"’In 


dx 


p(x) 


LP(0)J 


n+1 


p (0) cosAwxcosw^x 


(147) 
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A  similar  sequence  of  steps  yields 


^  <  1  .  Hence  ^ ^  decays  more  rapidly  than  ^-7^'  •  It  follows 

p(0)  -  IP (0) J  1  1  7  p(0) 

from  Equation  (147)  that  the  ratio  on  tin  right  side  o;  Equation  (]•'.(>} 
can  be  made  much  larger  than  unity  by  choosing  sufficiently  large. 

This,  of  course,  is  sinply  the  phenomenon  discussed  in  Section  III:  If 
cnc  target  Doppler  shift  is  large  there  will  indee-d  be  substantial 
clipping  loss  because  signal  and  noise  spectra  in  the  undipped  instrumen¬ 
tation  are  almost  disjoint.  We  are  here  concerned  with  any  additional  loss 
which  may  occur  in  Doppler  measurement.  Hence,  we  choose  w^  =  0  and 

attempt  to  set  bounds  on  A  for  that  case. 

n 

From  a  qualitative  point  of  view  otit  can  make  the  following  observa¬ 
tions:  Aw  would  be  chosen  so  as  to  place  w^  and  v  near  the  points 
of  maximum  slope  of  the  signal  spectrum.  Roughly  speaking  this  identifies 
Aw  with  the  half-bandwidth  of  the  signal  spectrum.  The  effective  duration 
of  is  given  by  the  correlation  time  of  the  signal.  Hence,  the 

maximum  value  of  Awx  in  the  effective  range  of  integration  is  of  the 

order  of  a  radian  for  D.  +  E.  and  less  for  D  4-E  ,  n  >  1  .  Therefore, 

11  n  n 

the  expression  for  D  +E  does  not  differ  greatly  from 

n  n  &  j 

C  f  |  ,  ,  n+ 1 

-f  ]  d’‘  [m5)|  • ' (1M’ 

which  is  (except  for  a  constant  indi  reiuknt  of  n)  the  same  as  the 

numerator  of  Equation  (100;.  In  the  1.  .\  press  ion  fer  F  ,  on  the  other 

n 


A-r>: 


hand,  the  effective  range  of  the  y  integration  covers  the  signal  duration 
and  over  that  range  &wy  could  go  through  many  complete  periods  (except  in 
the  absence,  or  virtual  absence  of  frequency  modulation).  From  the 
Riemann-Lebesgue  lemma  one  would  then  infer  that  2F^  is  small  compared 
with  D^+E^  s  except  possibly  in  the  absence  of  frequency  modulation.  One 
would  therefore  generally  expect  the  bound  on  given  by  Equation  (140) 

not  to  differ  drastically  from  Equation  (137)  . 

To  give  some  quantitative  support  to  this  line  of  reasoning,  consider 
once  more  the  Gaussian  pulse  with  linear  frequency  modulation 


lJT  K  2 

s(t)  =  c-  cos(wQt  +  —  t  ) 


Straightforward  compute*-'  >s  yield 


(Aw)  ‘ 


D  +  E  = 
n  n  4 


V2"  oT  ^ 


TT  1  4  (n+1)  ft 

—  —  e 

ft 


(150) 


(151) 


and 


2Fn  =  "T  V2  770T  fnfl  I 


_n_  _°X. 
n+1  2 


1  + 


1 


2  2 

2oT  ft  n 


(Aw)  ‘ 


(152) 


where  ft  is  the  "signal  bandwidth"  defined  by  Equation  (132) . 


Substituting  into  Equation  (140)  one  obtains 

2 

o. 


n  T 
n+1  2 


1  + 


2  2 
2  o.r  ft  n 


( Aw)  ‘ 


A  <  K  C  -I  l~r  - - 

n  —  n  In  \  n+1 


1  -  G 


oT 


1  +  2  2 

2cT  ft 


(Aw) ' 


(153) 
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The  signal  spectrum  has  maximum  slope  at  w  =  ±  /2  ft  .  With  thi 


2  2 

substitution  (Aw)  =  2ft  ,  Equation  (153)  becomes 


\  <  K  C  l/--~ 

n  -  n  In  V  n+1 


1  n_  2  2 

2  (nhl)  "  n+1  °T 
e 


l  2  2 

.  -  *  ‘  S'  “ 

1-e  e 


(154) 


One  can  readily  demonstrate  by  direct  computation  that  the  right 

2 

side  of  Equation  (154)  does  not  exceed  K  C,  for  any  value  of  o_  ft 

n  In  T 

and  any  n  ^  3  .  As  anticipated,  the  exponential  terms  [proportional 

to  the  ratio  of  2F  /(D  +E  )1  are  small  unless  ft  =  —  .  The  lowest 
n  n  n  1  oT 

2  1 

value  of  ft  is  reached  when  K  =  0  ,  in  which  case  ft  =  — t;  and 


A  <  K  C 
n  -  n  In 


(155) 


Thus  the  bound 

R  1  0.89  (156) 

remains  valid  for  Doppler  estimation,1  at  least  in  this  particular 
example  and  -  from  the  preceding  discussion  -  probably  for  most  cases  of 
practical  interest. 


'I'.'ote  that  we  are  working  with  =  0  .  Our  conclusions.,  therefore, 
refer  to  clipping  losses  aside  from  those  due  to  spectral  separation  of 
signal  and  reverberation. 
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Appendix  A 


Consider  the  probleir  of  detection  in  a  spherically  isotropic  ambient 
noise  field.  Here 


Pij(T) 


1 

2t 


ih 


X+T 

ih 


p(X)  dX 


(A-l) ' 


T-T 


ih 


where  p(X)  is  the  normalized  autocorrelation  of  the  noise  and 

d, 


ih 


ih 


(A-2) 


d_^  is  the  distance  between  the  i^  and  h^  hydrophone.  If  the 
system  processes  only  a  narrow  band  near  the  nominal  signal  frequency, 
one  can  write  as  before 


P (X)  =  p 1 ( X )  cos  wQ\ 


(A-3) 


Suppose,  now,  that  p^(X)  is  essentially  constant  over  the  interval 
t  -  tih£X^x+  for  every  pair  of  hydrophones  and  every  value  of 

x  .  This  is  the  narrow-band  assumption  discussed  in  detail  in  Section  IV. 
Then  Equation  (A-l)  becomes 


Pi  (t) 

p  ( X  )  =  T - 

1J  2lih 


;+Tih 


cos  w^odo 


T-T  .. 
ih 


sin  w  T.. 

0  i  h  /  \ 

-  p  (t)  cos  w  T 

W.T..  1  0 

0  ih 
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R.  A.  McDonald,  .  M.  Sc'.vjltheiss ,  F.  B.  Tuteur,  T.  Usher.  Processing 
of  Data  from  Sonar  Systems,  Vol.  I,  A-l,  Equation  3.  Soptembeir  1963. 
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Returning  to  Equation  (20)  and  expanding  the  sin  terra  as  in 
Equation  (22)  ,  one  obtains  the  following  equivalent  of  the  latter  equation 


T  T 

f  Jt  s(t)  I  dA  s(A) 


j  j  LL  ciJ  <t-A-h'"Tj) 

ij  -  1  +  K,  -2 - 0 _ L-i  i-i _ 

J  T  J  M  M 

[  dt  s(t)  i  d.\  s(A)  ?  r- .  .  ( t  —  A  -  a  .-it  ) 

J  J  (■ — >  ‘ — <  ij  1  J 

0  0  i=l  j=l 


M  M 


+  dt  s(t)  dA  s(A)  /  ^  tij(t"^_Ti+T-j) 


i=l  j-1 


4-  .... 


M  M 


dt  s(t)  J  dA 
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s(X)  L  L  ij(t_X_Ti+1i) 


i=l 


where  is  given  by  Equation  (25). 


(A- 5) 


Equation  (A-5)  is  of  the  same  form  as  Equation  (32).  Hence,  if  one 
r.g'i.n  postulates  a  signal  of  the  form  (31)  one  can  pursue  an  argument, 
entirely  parallel  to  Equations  (33)  -  (38)  .  The  topic. a.  term  of  Equation 
(A-5)  now  becomes 


r  T  _M 

dts1(t)  f  dAs1(A)p“(t--A))  ) 


A  =K  C,  - 
n  ti  In 
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Vij 
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slw0^'ri_T  )+i,(t)-ij(A) 
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d ts  ^  ( t )  I  dA  ( A ) p  ^ ( t  —  A )  j 

o  -=i  -Tl  1  -i 


— - COS  j  Wq  (x^  — j)  +  5.(t)  ~d(  A  )  ] 


(A-6) 


In  deriving  this  equation  p  ^  ( t- > -t  ^  )  has  hen  replaced  with 


A-.V 


p^(t-A)  ,  invoking  once  more  the  narrow-band  assumption,  that  the  correlation 
time  is  large  compared  with  sound  travel  time  across  the  array.  The 


definition  of  C  is  given  in  Equation  (35)  . 

If  we  consider  an  array  confined  to  a  plare  and  a  target  producing  a 
plane  wave  parallel  to  the  plane  of  the  array  (remote  broadside  target) , 
then  r ^  for  alJ  i  and  j  and  Equation  (A-6)  reduces  to 


A  =h  C, 


M  M 

L  L 

i=i  |=i 

1  sinw^^  \ 

l  Vu  i 

M  M 

sinw»T  .  , 

\“ 

0  ij 

L  , 

V  T  ,  . 

i=l  j=l 

o  i  .l 

ratio  of  integral: 

j  dts^ (t)  j  dXs^  (A) p^(t— X ) cos f<J>(t)  — 4>(A) 
0  0 


dts  (t)  j  dAs.  (\)p  (t-A)cos[o(t)-ij(X)] 


(A- 7) 


therefore  has  an  upper  bound  of  unity.  This  upper  bound  can  be  approached 
arbitrarily  closely,  for  instance,  by  choosing  ^(t)  =  0  (no  frequ'  r.ey 
modulation)  and  taking  r.(T)  ?  1  for  1 1  j  <  T  (narrow-band  noise). 


Thus 
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n  In  M  M  sin  u nx  . 
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The  upper  bound  can  be  approached  as  closely  as  desired  by  proper  choice 
of  ( t)  . 

Consider  now  a  planar  array  constructed  from  .quilateral  triangles. 
Figure  .'1  shows  the  most  elementary  version  of  such  an  array,  with  only 
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Figure  A1 


For  the  next  larger  regular  hexagon  (Figure  A 2,  nineteen  hydrophones). 

ti'.e  maximum.  A  is  2.53  K  C,  .  Going  to  >.  still  larger  regular  hexagon 
n  n  In 

(seven  phones  along  a  diameter  for  a  total  of  thi i ty-sevon  phones),  one 

arr ’ ves  at  a  maximum  A  of  3.02  K  C,  .  In  each  case  A,  ,  A_  .... 

ti  n  in  5  / 

are  onlv  slightly  L.rget  than  A  (  b.cuse  only  the  terns  i  =  j  contribute 
significantly  to  the  numerator  rf  Equation  (<\-l3) .  Using  A  -•  3.02  K  Cj 
and  following  the  procedure  of  Iquntions  (3d)  (43)  one  obtains 

F  •  0.74  (A-9) 


Thus,  the-  clipping  i  v,  v.  ir".  iner.asij  r  .  .0  l  1 1  ror.i 
uncorrelated  phor.es)!’  No  ttr.pt  w  >  s  t  <•  •  1 1  u  •  tv 

structures,  but  it  .ppe  ars  net  ur.re  ason.ah !  •  that  stiLl 
losses  might  he  vneount-  t.u.  Ih.  physic  a  I  :•  as  fur 
phenomenon  are  c  1*_  i r  :  it.  .  n,,  .  .. ..  i. ... i  pit 


.1  db  ft  r 

or.  complex  irray 


larger  clipping 
the  observ.d 


c  ireful  1 


in  tern- 


i  L VU  1 < 


incut  1,.:  1 


of 


u  ratio. 


selected  to  yield  high  negative  noise  correlation  between  adjacent  phones. 
This  reduces  the  effective  noise  power  in  the  unclipped  case,  but  is 
ineffective  in  the  clipped  case  because  the  generated  harmonics  do  no  retain 
the  same  phase  relations  as  the  fundamentals. 


Figure  A2 
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Appendix _H 


In  order  to  exhibit  the  influence-  of  the  t r on.  -nitter  pattern  .n  .  >e 
narrow-bend  approximation,  return  to  Equation  itwj  end  evaluate  R  at 
the  point  t  -t\+t  *  Using  Equation  (R4) 
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Let 


x  =  (a-a{j)  +  * 


( B— 2 ) 


Then,  using  Equations  (68)  and  (69)  one  obtains 
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for  narrow  beams  on»  obt  -.ins  from  Equation  vhi) 


<  *  h  ,  ,  (v„ -$n) 
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whv  re  tr-  .  and  y .  .  ;i\  defined  bv  Equations  tRi)  and  (-84).  It  is  a 

i  J  hi 

•  lr.ple  matter  to  demons ti  .t  to-  t 


1  nd 
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Hence , 


max  | a  -  a  I  <  max  1 0  -0  |  +  nax|^„  -  v  | 

1J  9  U  d>„  u 

Approximation  1)  tp.  27 j  is  at  least  as  good  as  before.  In 

place  of  approximation  2)  one  novr  has 
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Thus ,  one  makes  the  implicit  assumption 
,  ,  d , 


I  max  0  (x  -  -r) 


—  [max (a  -  a  )j  <<  1 

C  1 J 
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Using  Equation  (B-6)  this  becomes 


r  u  .  .  , 

| max  $  (x  -  ^-)  |  -1-— (max  1 9  -  0  |  +  max ]  i,  -  0  J  j  <<  1 

1  :  0  *■  u  U 

The  maximizations  in  0^  and  6^  extend  over  the  width  of 
transmitter  pattern.  Fc”  a  maximum  frequency  deviation  of 
a  maximum  distance  between  hydronhoncs  of  20  ft.,  Equation 

0,4'i;  (sun  of  3  and  b  pattern  half-widths) 


(B— 9 ) 


the 

50  cps  and 
(B-9)  reads 

<<  1  (B-10) 


The  pattern 
need  not  be 
restrictive 


half-widths  are  measured  in  radians.  Hence,  the  patterns 
very  narrow  before  Equation  (B-9)  becomes  substantially  less 
than  its  equivalent  in  Section  IV. 
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Appendix  C 


Single  Frequency  Pattern  Functions 

Consider  a  transmitting  array  consisting  of  f  omnidirectional  point 
transducers  in  an  arbitrary  (but  specified)  geometrical  arrangement.  We 
wish  to  find  the  angular  distribution  of  power  in  the  far  field.  The  basic 
geometry  is  shown  in  Figure  Cl  and  is  basi  sriy  t;n  s.av  as  t ha u  of  Picur .  2. 


.  P 


Figure  Cl 


Ignoring  spherical  spreading  losses 
in  Equations  (61)  and  (62)  |  tin 


(which  are  not  part  of 
pow.  r  at  point  P  is 


the  coefficients 


(C-l) 


transducer 


th 


where  t^  is  the  delay  of  the  signal  applied  to  the  j 

th 

(steering)  and  i  ^  is  the  sound  travel  tine  from  the  j  transducer  to 
point  P  .  The  pattern  function  g(  )  measures  the  angular  dependence- 
of  power  and  is  therefor^  directly  proportional  to  (C-l) . 

From  Equation  (65)  one  obtains 

d, 

=  tg  — _  [sin  0  sin  0^  cos(tf<-(^)  +  cos  fi  cos  0 ^ ]  (C-2) 

where  tg  is  the  sound  travel  tine  from  the  origin  to  point  P  and 

t  h 

dj  is  the  distance  of  the  j  transducer  from  the  origin.  If  the 
array  is  steered  in  the  direction  of  (0^,  Oq)  then 


d. 

t .  =  — [sin  9  sin  0,  cos((t'r,  -  $.)  +  cos  0n  cos  0,J 
J  c  0  j  0  j  0  j 


(C— 3) 


If  the  pattern  is  confined  to  small  angular  deviations  from  (0^,  0Q) 
one  can  approximate  (C-2)  by  the  first  terns  of  a  Taylor  series 


h  a  c0  -r  s  +  V9  -  V  +  V4  -  Vj 


(C-M 


where  ,  Yqj  are  analogous  to  the  parameters  a^_,  , 

defined  in  Equations  (80)  -  (82).  From  inspection  of  Equations  (80) 
and  (C— 3 ) 
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Now,  substituting  Equations  (C-4)  and  (C-5)  into  (C-l) 
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Hence,  the  Fourier  transform  of  g(  )  is 


G  (u ,  v) 


f  t  /d */d,/>'[<VV*+SrVy]e.i(u*+,y) 

j"l  k=-l  -® 


N  H 


-ti  oj  ok 


ixr<en,-8nJw-u]  “  iy[(Y0j-Y0k)w-v] 


i=l  k=l  -« 


N  N 

=  L  L  6lu"w(f30j"60k}^ 

i=l  k=l 

Thus,  G(u,  v)  is  a  non-negative  function.  This  result  clearly 
remains  true  if  one  considers  the  pattern  function  defined  by  the  total 
power  over  some  frequency  band,  for  the  w  integral  of  (C— 7 )  is  clearly 
non-negative . 
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Summary 


The  problem  of  optimum  bearing  estimation  Is  discussed  for  riu.  ..a* 

possible  case.  The  receiving  array  consists  of  two  hydrophones.  .Sign..]  and 
noise  are  stationary  Gaussian  random  processes  with  zero  means  and  spectra  of 
the  same  form.  The  noise  at  out:  hydrophone  is  statistically  indeper.  (cr.l  from 
that  at  the  other.  The  foil; -.hop  rear.'!:  ...  .  ;  a  J .  -d . 

1)  If  the  output  s  ignal-to-no  i  •  r.nlo  is  «.  ..ough  to  perm,t  accurate 

bearing  measurements,  the  rms  bearing  error  of  a  simple  cross-correlato- 
(with  pre-whitening)  reacb.es  the  Crame r-Kao  lower  bound  for  all  input 
signal-to-noise  ratios.  Hence  the  cross-correlator  is  an  optimal 
bearing  estimatjr. 

2)  In  terms  of  the  input  signal -to -noise  ratio  (S/N)  the  rms  bearing 

error  of  the  cross-correlator  varies  as  (S/N)  ^  for  (S/N)^<  1  and  as 

(S/M)  for  (S/N)  ••  1.  Thus  the  optimum  bearing  estimator  has 

characteristics  normally  associated  with  a  coherent  (incoherent)  device 
for  high  (low)  input  signal-to-noise  ratios. 

3)  The  rms  error  of  the  cross-correlator  varies  linearly  with  the 
correlation  time  (Inverse:  bandwidth)  of  signal  and  noise.  It  varies 
inversely  with  the  half  power  of  the  tlme-bandwfdtn  product. 

A)  The  rms  bearing  error  of  a  2  element  split  beam  ..racket  is  equal  to 
that  of  the*  cross-correlator  if  the  phase  shift  between  beams  is 
achieved  by  a  pure:  differentj.it  -r.  If  a  pure  1  phase  shift  Is 
used  in  place  of  the  differentiator  there  is  a  small  loss  in 
performance,  cm  ivale.nt.  t  o  about  0.6  db  ol  Jnpu*  s  iga a  1-to-nolse 


rol io. 


I. _  Introduction 


The  purpose  of  this  note  is  to  correlate  and  extend  certain  previously 
reported  results  on  opt  Ltnum  and  sub.-.j-t  fir  err  ,  ■  intimation.  The  stimulus 
was  provided  by  a  recent  paper  by  Middleton‘S  using  a  very  different  approach 
and  obtaining,  in  part,  different  results. 

In  the  following  analy*. :  <5  the  problem  of  a;  1 1; ..  1 1  i  t  y  is  approached  by 
using  the  Cram?  r-Rao  i in.  quality  to  s.  i  a  lower  hound  on  the  attainable  nns 
bearing  err  r.  Re.ili? ability  is  then  established  by  describing  an 
instrumentation  which  actually  roaches  the  lower  bound.  Only  the  simplest 
physical  situation  is  considered.  Signal  and  noise  are  assumed  to  be 
stationary  Gaussian  random  processes  with  zero  means  and  spectra  of  the  same 
form.  The  noise  is  assumed  to  be  statistically  independent  from  hydrophone 
to  hydrophone.  The  observation  time  is  long  cc  arid  with  the  correlation 
times  of  signal  and  noise.  In  most  of  the  discussion  the  receiving  array 
consists  of  a  single  pair  of  omnidirectional  hydrophones. 


D.  Middleton,  "n  < 
be. terminis ti c  and  Kar  11 
Society  of  America,  7  ird 
treatment  of  the  s  ms  ’ 
Acoustic  Signals  by  Fa  e 
’.‘e-port  ( S  ,  .  r , .  .  lip:  .. 


iir.um  ,.r.d  Subopt imuir  5k, a:  ins  i  e»  imation  for 
Signals  in  Norma!  ;.ei'i  1  i.  hs,1'  Paper  Ef  7 ,  Acoustic 
hoi.  ting,  April  l°67,  York.  For  a  mori  detailed 
■  1  •  '  .  '•>!•!''  ■  -r. ,  1.x t  r~  ’Lion  of  Random 

ivers  with  F1  ■  ,  t  r  i  ;  ut .  a  •  kner.l  s , "  Rav theon  Company 


.  !  nt  ,  c  : 


i  u  u . 


, ;  <.  L 


II.  Lower  Bound  on  RMS  Bearing  Lr ror 

If  the  receiving  array  consists  simply  of  a  pair  of  oranidf  a.  ct  •  ; 

hydrophones  the  problem  of  bearing  ..t  l:  t  i.-m  i  .  qi.Ivalcnt  t<  the  {  roblc-o 
of  estimating  the  signal  delay  between  the  two  hydrophones.  Let  the  true 


and  estimated  delay  be  -S 


■no 


luctivciy.  I'cr  l 


cjsl  of  signal 


and  noise  processes  with  tit  propvrti  oi  m  wh  1  i  noise  t  •  d  Jn 


frequency  to  0  <_  f 


Mclionn.ld'’  i 


calculated  the-  Cr  imJ'r-Rao  lower 


bound  on  (6  -&*) 
o 


(6  -6*)2  >  (S+V  <s+N2>  "  S 

o - - - 


JL _ _J _ 

HW2  _r2W T  -  iog(2WT-l)  -  0.3772 
3 


(1) 


S  is  the  signal  power,  1!  ,  the  noise  power  at  phones  1  and  2 


respectively  and  T  the  observation  time.  If  N 
Equation  (1)  reduces  to 


“a 


N  and  WT  •>  >  1  , 


(t-6*)  > 


-1 _ (2  -N-  +  M-) 

-  9  ?  V-  g  2 

8n“  W  (WT)  S 


(2) 


In  order  to  Irons  1  ‘to  Equ.it  ion  (2)  Into  a  lower  bound  on  bearing 
accuracy,  consider  the  geometry  of  Figure  1.  If  the  L  rget  distance  r 
is  very  large  compared  with  the  spacing  d  botva  e-n  hydrophones,  om  can 
write  to  an  excellent  approximation 


5  1  II 


(3) 


A .  L .  Levesque  ,  R  .  A . 
of  Data  from  Sonar  Svs*  •  -i«  .  V  t  [ 


.  ’  lie  I 

n  I  i  x  1  . 


I  si 


i  i  ■  ■  r:  . 


c  is  Che  velocity  of  sound  in  water. 


x  target 


tin':  I 


/van-  1 
o 


If  tin.  true  bearing  is 
is  related  to  !  . 


n.  n  tlit  ustimt’en  error  0*  -  0 


<J*  -  f 


(,  in  •  • A  — s i  i 


sin  1  j  ( • )  ■*'  -  *  )  cor-  4(^:V+t'  ) 

O  0 


In  practice,  one  n,  ■.  one. -mod  with  t  •  uun.-iit'l  v.j.'ue  or  the  rms  tracking 
error  only  if  it  ;  iu:ti  s;  :  i  1  ,  :  ■  -  "  dots  not  exceed  p 

snail  fraction  of  a  r.id.t an  vi •  h  any  significant  probability .  In  that 
case,  one  can  .apprcxir; 


'4) 


(3) 


and,  except  for 


extremely  close  to 


_n 

2  ’ 


cos  -5(0*50  )  '  cos  ■' 


if,) 


Substituting  these  approximations  in  Equation  (A)  one  obtains 


6*  --  6  ~  (0*-b  )  -  cos  (3 

u  or  O 


\T, 


Hence,  the  standard  deviatto;  of  ( 0*— 6  )  has  the  lojcr  hound 


b(6*) 


(6*-9  )' 
o 


■  3  c 


2  /2  ,  «  / wl  d 


cos  0 


o 


1  s2 


(B) 


This  result  generalizes  tn  trivial  fashion  to  not. -white  signal  and 

noise  spectra,  as  long  as  the  ratio  of  the  two  spectral  functions  is  a 

constant1  over  the  processed  frequency  band  0  <  f  W  .  For  then  one  can 

regard  the  output  of  each  hydrophone  ns  prowhi  to-tied  by  an  appropriate 

linear  filter,  prior  to  further  processing.  The  (invertible)  linear 

filtering  operation  can  clearly  not.  have  any  effect  on  the  minimum 

a 

attainable-  mean  square  error.  Thus.  Equation  (11  r. -mains  ''did. 


In  practice  this  condition  is  often  at  least  approximately  true, 
because  signal  and  noise  spectra  art  shaped  by  the  same  hydrophone  id 
receiver  character  is t i c j . 

2 

This  argument  does  not  prove  forma!  iy  that  tin  "raia  .r-Rao  low  t 
bound  might  not  be  lowei  for  nor-w'nite  spectra.  Howl  cci ,  it  does  piovi 
that  any  such  .lower  value  cannot  Oe  niJlzabK.  The  optimum  estimator 
after  prewhitering  cannot  achieve  a  p  rfoninnco  better  Mian  the  right 
side  of  Equation  (P)  .  hut  the.  optimum  a  imnt  ur  voi  Id  include-  linear 
filters  to  shape  the  spectra  to  the  most  desirable  1.  arm.  Therefore,  no 
realizable  estimator  working  with  Uu  initial  spectra  can  improve  on  the 
lower  bound  of  Equatin',  (at. 


III.  An  Optimal  Instrument  ntion  :  Hie  Cros s-coitl  I  '.tor 


When  only  two  hydrophone  &  nr...  available  nr.  obvious  instrumentation 
for  delay  measurement:  (and  hence  beat  ing  r.eusur  nit)  Is  the  simple  cros 
correlator  shown  in  Figure  2. 


ohene  1  o— • 


x  ,  ( I  -  u-  )  +  .1  v  \  ) 


y(t) 


low  pa.  s 

filter 


z  (t 


/mu  H  -  p  1  !•  r 


phone  2  o- 


x„(i) 


delay  t. 


1 


=  S  ( t )  +  n  ( *  ) 


Figure  2 


Tne  system  output  Is  the  short- time  cros  .-correlation  (nvi t aged  over  the 
smoothing  tine  of  the  filter)  of  the  two  hydrophone  outputs,  delayed 
relative  to  each  oilier  by  i .  .  At  uy  giv.  n  i  i-i.  t  ,  z>t;  L  )  is 

A 

computed  for  all  possible  value  s  of  C.  .  The  v  iue  t  ot  t^  which 
maximizes  z(t;  t  )  1  .  <i.  sign.it.  n  a  th  i  it  mr  irieous  estinat.:  of 

signal  delay.  The  healing  , : ;  t  i ;  t.  is  then  eht.  in.d  from  an  obvious 
analogue  of  Equation  . 


The  maximum  of  z(t;  t^) 


occurs  at  a  point  where 


3  Z 


at 


! 


(K 


Thus  the  rns  error 
of  this  derivative 


in 


This 


is  equal  to  the  rms  fluctuation  of  the  null 
fluctuation  Is  g  I  vt  n  by 


~V 


at . 


t 


1 


(ID 


D  (  )  denotes  the  standard  deviation  of  the  bracketed  quantity. 

The  output  y(t)  of  the  multiplier  in  Figure  1  is  given  b/ 

y(t)  =  fs(t-<5Q)  +  n  ^  ( t )  j  [s  ( t  —  t  ^ )  +  n.^t-t^J  (12) 

If  the  noises  at  the  two  receivers  are  statistically  independent  of  each 
other  and  of  the  signal  (and  have  zero  means) 

z=y=Sp  (t-6)  (13) 

s  1  o 

p  (x)  is  the  normalized  autocorrelation  function  of  the  signal, 
s 


T)e  are  assuming  that  3z/Dt  has  only  ntu  z-  ro  in  the  neighborhood 
of  the  correct  value,  t^  =  6  .  For  s ignal -to-noise  ratios  high  enough  to 

permit  tracking  with  reasonaSle  accuracy  this  should  be  a  good  assumption. 

In  evaluating  sljpe  and  standard  deviation  at  t.  =  6  ,  wc  imply 

furti  that  the  null  of  3z/3t^  fluctuit  .  by  less  than  tB.e  correlation 

time  of  the  signal.  Examination  of  Equatir  (Jh)  reveals  that  this  is  true 
when  the  output  slgnal-to-noise  ratio  is  Large  compared  with  unity.  This 
is  the  only  condition  under  which  the  bearing  accuracy  problem  has  much 
practical  interest. 


I  ’ 


Differentiating  liquation  (13)  twite  with  spect  to  t^  mid 
evaluating  at  t  =  ,  one  obtains 


u 


v  '  1 1  ) 


i ! 


(14) 


t  . 


If  the  lowpas  i  i )  t  ..  i  .  gup 
the  output  7.  (  •  .  'ey 


.  wi  tg lit.  mg  mnot  ion  h(o)  , 


«•  ( L ) 


•  t  :)  ( t  a  ) 


(15) 


hence , 


2-‘~ 
•i  t . 


I  i  h  (  • ) 


1  lllL-vi 


(16) 


We  must  <•  onslder  i  f  i  I  •  *  which  snoot  he  a  tie  put  over  the  past  T 
seconds.  Such  n  i  i  ;  ■  i  ..  ;  tin  w.  lplit  i  nr  ft  •„ 1  Ion 


(17) 


t  1  St  V' 

Li  T  nut1;  1  :  r  i '  i  tlr  tie  ion  '  i  n  tin,  of  signal  and 

noise,  it  is  a  simple  ratter  t  liv.nristr  ■  It,  po>  '  .  10,  liquation  (9)] 

that 


—  I  = 


1  L' 


;  -  R.u  W")  |  dx 


(18) 


it. 


V1 

s'i 


Is  the  nutoi 


1 1 . 


•  r  •  t  ton  t  u.  ’  1  ii  1 


From  Equation  (12) 


at. 


-  [s(t-6^)  +  n1(t)]  f s ' ( t - t 1 )  +  (t-t.^)] 


(19) 


Hence, 


V(,) 

St. 


E(  [s(t-6o)+n1(t.)J  [s (t+T-6^)+n^ ( t+x)]  [s'  (t-t^+n^ (t-t^)]  [s'  (t+T-t^+n^t+x-t^)]  ) 


(20)- 


Twelve  of  the  sixteen  terms  in  this  average  vanish  immediately,  because 
they  contain  one  of  the  noise  components  only  once.  The  remaining  terras 


are 


R  (x)  =  E{ s ( t -6  ) s ( t+x-6  )s'(t-t  )s  (t+x-t  ) } 

iiL.  o  o  1  1 

3tl 

+  SNps(x)pn' (x)  +  SNp^'fx)  Pn(t)  +  N2  Pn(t)  pn'(f) 


(21) 


Px(x)  designates  the  normalized  autocorrelation  function  of  the  random 
process  x(t)  .  Both  noise  processes  are  assumed  to  have  the  same  auto¬ 
correlation  function 

Since  the  signal  process  is  Gaussian,  the  first  term  of  Equation 


pn(l)  ' 


(21)  can  je  expressed  in  terms  of  second  order  moments.  Thus 

Ri^(r)  =  s2Ps(t)Ps,u)  +  Rss,(6o_t:l)  +  Rss,(6o_tl+T)  Rss,(60_trr) 


+  SN[p  (x)  p  '  (x)  +  p  !(x)  p  (x )]  +  N2  p  (x)  p  '  (x) 
°  n  <=  ri  n  n 


(22) 


hi 


stands  for  the  expectation  of  the  bracketed  quantity. 


B-9 


Here  R  (t)  is  the  (unnormal  .tied)  crons-correiation  of  x(t)  and 
y(t)  . 

There  remains  the  computation  of  the  various  correlation  functions 


/  \  _  E(s  '  (t)s '  (t+t )  )  lirn  ^[s(t+At)  -  s(t)j  |.s(t-K-At)  -  s(t+r)'J 

s  "  s  s  l.,)! 

.  ?-(j  (t  )  -  .  (  i-*-.'  t  i  -  p  ( >  -At ) 

lim  s  s  a 


At-0 


(23) 


(it)' 


Now  expanding  the  ljst  two  terras  into  TayLoi  series  about  the 
point  \  • 

f  2  7 

p„,(t)  =  ^mn - /  2,  (t)  -  |p  (0  +  O  ’(t)AT  +  P  "(T)  -yy  +  0(At)2] 

s  At-0  {Lt)>  S  s  s  2 

1  2  ,  1 

-  If  (T)  -  1  (  : )  A  t  +  p  m(t)  ~^r  +  o(At)2!  ) 

S  s  S  2  j 

(24) 

2 

Tne  r.cfation  o(At)*'  indicates  that  tiie  remainder  term  in  question 

2 

approaches  zero  faster  than  (At)  as  At  *-0  . 

!iow  carrying  out  I  hi  limit  Inp.  operation 

.  ,()=-;';(!)  (25) 

b  fj 

By  campletcly  analogous  computations  one  obtains 


,  (r)  =  -  "  (a) 

i!  n 


(26) 


and 


K  .  ,  (t)  =  S.  '  (  r )  =  -  R  ,  (-t ) 
t»  s  s  s  s 


(27) 


In  the  absence  o •  DC 


and  other  dote rm L ni s c ic  components  of  signal 


and  noise,  all  of  the  correlation  functions  and  their  derivatives  tend 
to  zero  as  i  -*■  <*>  .  Hence  from  Equation  (22) 


R.  <*>  -  -O  “  Sp  '  (6  -t  ) 

dy  S3  0  1  sol 

3t, 


(28) 


Now  substituting  Equations  (22)  and  (25)  -  (28)  into  Equation  (18) 


n“  '  '  t  di(E2[ps(''»s"o)  +  Ps'O-Vh)] 


„.a 


+  SN  [p  (t)p  "(t)  +  p  (i)p  "(t)I  +  N  p  (i)p  "(t) 
L  s  n  n  e  J  n  n 


Using  Parseval's  theorem  and  the  real  translation  theorem 


I 


(29) 


n2  3z  2jl  ,  2J  2  2, 

D  [JTJ  T  J  dWM  S  Es  (w) 


-  2jw(6  -t  )-,  ?  2  l 

1-e  j  +  2  SN  gs(w)gn(w)  +  N  gn  (w) j 


(30) 


where  g  (w)  and  g  (w)  are  the  normalized  spectral  functions  of  signal 
s  n 


and  noise  respectively.  Note  that  the  first  term 


ii 


2  2 

dw  w  S  g  (w) 
s 


r  2jw(6  -t  ) 

1-e 


oo 


exhibits  the  effect  on  the  output  variance  of  a  disadjustment  in  t^  . 

For  t.  =  6  this  term  vanishes  and  one  obtains 
1  o 


CO 


B-l  1 


Noting  that 


(33) 


we  can  now  substitute  liquations  (14)  am!  (32)  into  Equation  (11)  to 
obta in 


/  2  ••  2 1  2  I 

/  —  <1w  w  ]  2  SN  g  (w)g  (w)  4  N  i"  (w)  ! 


s  r. 


(34) 


S  I  dw  w“g  (w) 
I  s 


Consider  now  the  white  spectra 


g  (w)  =  g  (w)  =' 
s  n 


;  4M,r  ’ 


jvl  2’:W 


(35) 


0  ,  |wj  >  2t>W 


Straightforward  evaluation  of  the  Integrals  in  liquation  (34)  yields 


1  2 


.(41  ) 


(36) 


which  is  identical  witn  the  square  root  of  the  right  side  of  Equation 
(2.).  Translation  of  delay  error  into  bearing  i  rror  proceeds  as  in 
Equations  (j)  -  (a),  v : .  i.iisg  the  result 


/ 


0(4*) 


’5  e 


.2 


■v'l  e  os  f 


v  r 


(37) 


i .  i 


Thus  the  instrumentation  of  Figure  2  actually  attains  the  Cram&r-Rao 
lower  bound.  The  value  of  t^  which  maximizes  z(t,  t^)  is  therefore 
the  optimum  bearing  estimate.  It  is  obvious  by  inspection  that  it  is  an 
unbiased  estimate,  as  demanded  by  the  formulation  of  the  Cram&r-Rao 
inequality  used  in  deriving  Equation  (2)  . 

While  the  last  part  of  the  argument  has  been  carried  out  for  white 
signal  and  noise,  a  trivial  modification  of  Figure  2  generalizes  tba  result 
to  Gaussian  signals  and  noises  with  arbitrary  spectra,  as  long  as  the  ratio 
of  signal  and  noise  spectra  is  a  constant  over  the  processed  frequency 
range.  In  such  cases  one  simply  inserts  a  prewhitening  filter  after  each 
hydrophone,  thus  reducing  the  problem  to  the  one  just  treated. 

It  is  interesting  to  observe  that  the  rms  error  of  the  optimum  tracker 
varies  as  the  inverse  first  power  of  S/N  for  small  input  signal-to-noise 
ratios  (the  typical  behavior  for  rtnt  processing)  whereas,  for  large 

input  signal-to-noise  ratios  it  varie  <•  the  inverse  half  power  of  S/N 
(the  typical  behavior  for  coherent  processing).  For  large  input  signal-to- 
noise  ratioe  or.e  can  reason,  of  course,  that  the  waveshape  at  each 
hydrophone  is  a  good  approximation  of  the  signal  waveshape.  Thus,  one  has 
something  akin  to  knowledge  of  the  signal  waveshape,  the  distinguishing 
characteristic  of  coherent  operation. 


b-  i 


IV.  Conventional  Split  Beam  Trackers 

An  Instrumentation  frequently  used  in  bearing  estimation  is  the  split 
beam  tt acker,  an  elementary  version  of  which  is  shown  in  Figure  3. 
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Figure  3 


This  configuration  has  born  analyzed  In  d<  t,  II  it!  Report  No.  29.  Omitting 
the  interference  ( 1 1  u  quantity  of  primary  interest  in  Report  No.  29),  but 
retaining  the  sign  1  1  n  i  :  e  rr.  i  f  lu>  tuition,  one  obtains  from 

Equation  jPcport  ..  ,  :lr..r  a  lew  steps  of  computation 
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(38) 


on  target 


oz  is  the  output  variance  and  M  is  the  number  of  hydrophones  in  each  half 
of  the  array.  All  other  symbols  retain  their  earlier  meaning. 

Using  again  the  white  spectra  of  Equation  (33),  one  obtains,  after 
evaluation  of  the  integrals 
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From  Report  No.  29,  Equation  (65) 
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For  direct  comparison  with  the  optimum  bearing  estimator,  consider 
M  =  1  (array  consisting  of  two  hydrophones)  and  take  the  ratio  of  Equations 
(37)  and  (41) 
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Thus  the  split  beam  tracker  performs  almost  optionally:  The  ratio  of  rms  errors 
is  0.87,  equivalent  to  about  0.6  db  of  input  s ignal-to-r.cice  ratio. 

If  in  Figure  3  one  replaces  the  pure  phase  shift 

F{Jv>)  -  — (43) 

J  |W" 

with  pure  differentiator 

H(  jv)  «=  |w  (44) 


Equations  (39)  and  (40)  assume  the  form 


2tt  W 
3T 


j  2SN  M3  +  tlV  | 


(43) 


on  target 


and 


3_Z 
3  0 

on  target 
hence,  1  or  M  -  1 


4p‘I-r  _  3  d 

-  -  -r—  S  M  -  cos 


D(  0) 


'3  r 


?./2  n  W  /WT  d  cos  0 


1  _ 

~  8 

o  N 


(46) 


(47) 


This  is  identical  with  Equation  (37)  and  with  the  Cramer-Rao  lower  bound. 
Thus,  at  least  in  the  simple  rase  under  discussion,  the  conventional  split  beam 
tracker  actually  •„  Moves  tne  s  :  i our,  possible  pe~f ormance . 
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The  obi^ct  -r  t.l.a :> 


. ,  ate  'At1'  effr-ct  of  li-on  .  .  ••od  noise 


sources  on  tV  per forr..s t  the  «  f]  i kel  ,  Wot-re ‘.ii, )  detector . 

In  a  prciciv'  r,  r  :  icr.s  ’>•  ;•  ‘In  ;  ei  fnri'.a:  -  e  loss  of 

J  tKe’ii  hr<  d-r  .*  '  :>  ••<  *  r : . r  ,  ■  i.y  ■  •  .  t .  ’  re:  i  .  r  and 

a  coni  o:u.'  '  a  r  •••.  .•  .  1  .■  •  ! .  ’  >  c  present  :•  p  u'f 
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sources  that  are  not  r  tr-rlv  .oral l//. -uei  sources  can,  presumably,  be 
represented  by  a  larre  number  of  uj.esrly  a pa cud  p^int  sources. 

The  notation  ,sed  f  o !  loves  •  t  ''  :•  >• .  •  i  et.  at.  j  ?[  ,  si;.  this  per¬ 
mits  inoor'  ar.t  rirt  ;  ;  .  1  ns  •  v?r  '  l.e  r  '  a  :  -j;;-  d  previously.  For  1  he  sake 

of  completeness  the  .  rest.  •  for  '  ■  .mod-  •  1  o  ;  rc-'-'sscr  >re  rederived 

here  and  a  brief  comp -nr;  scr.  bo'  v  th'>  .-•!  ;  >:rl  no-,-  notation  is  given. 

Unfortunately,  n,  spite  of  tuu  simplified  notation  the  fir.  result  is  still 
difficult  to  evaluate.  /  proximate  ..It  show,  ng  that  the  nor  forma  nee  loss 
due  to  point  noise  sour  .  1 ui valet  to  the  loss  of  -no  hydrophone  per 

point  source  can  be  ot-urt  v  >  nrunaii  .1.  ,f  ali  of  ’tic  •  -  ms  causing 

coRiputatior.nl  dil  s  jcui  tin...  r.c;  since  t  0  terns  also  contain  all  the 

inforr.atj.cn  a  nr  r  ; :  >■  ••  ..•  j..e  .  ircc.e  this  approxi¬ 
mation  is  ia;r,i,.'  :  '  ;  .  .  c  nurrer  .f  localized  noise 

sources  is  s:  n;  J  ■■  ■  ;  •'  r-^n  p  <  :  other  <r.  f  rom  ‘  lie 

target  direction.  ;...  '  >:  \  •  obta  ■  for  tr.  til 

'  '  ’’I  i  ’•  —  d.i  .i  ■  .  .!  ;  l:'.l  ^  ’•"* 
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azimuth  has  approximately  the.  same  effect  on  detectability  as  a  single  point 
source  of  the  same  strength  located  at  the  center  of  the  distributed  source. 

For  large  anisotropic-to-isotropic  noise  ratio  a  similar  result  can  be  demon¬ 
strated  only  if  the  anisotroroic  noise  source  extends  only  over  a  much  smaller 
angle . 

In  order  to  get  seme  idea  of  performance  loss  in  cases  not  covered  by 
these  analytical  results  computations  were  also  performed  on  the  digital  computer. 
Although  these  computations  are  not  conclusive  they  indicate  that  the  analytical 
results  for  distributed  noise  sources  at  low  nni3otruu±o-to-isotropic  noise 
ratio  can  probably  be  extended  to  largo  auj  u-obroi/ir.-to-d  notix/pio  -ratio. 


II .  Basic  Analyst s 


Assumo  that  the  array  consists  of  M  hydrophones,  and  that  the  received 
signal  at  the  hydrophone  is  x^(t)>  Then  if  the  spectrum  of  x^(t)  is 
limited  to  frequencies  below  W  cps,  and  the  x(t)  are  observed  over  an  interval 
T,  such  that  WT  »  1,  (t j  can  be  expanded  in  a  Frailer  3erles: 
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(1) 


where  the  x^(n)  are  complex  Fourier  coefficients  satisfying  x^(-n)  •>  x*(n)  and 
where  the  asterisk  stands  for  complex  conjugate.  Then  all  the  available  in¬ 
formation  about  the  signals  received  by  the  entire  array  is  contained  in  the 
set  of  vectors 
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2^  (n )  j 
^(n) 


(2) 


Following  Bryn  13  I  or  bdelblute,  Fisk,  and  Ktnnison  1 2  |  ,  we  assume  that  X(n ) 
and  X(m)  are  statist,- rally  independent  for  n/  -  n..  .‘'iiipjMjue  that  the  signal 
x_,  (t )  received  at  tne  i^  hydrophone  consists  of  .signal  and  noi.!x>}  then  wo  lot 
the  signal  be  giv . n  by 
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so  that  tlie  signal  at  all  hydrophones  Is  topi  esen  ted  by 
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Here  again  we  assume  the  Y(n)  tc  be  independent  from  Y(m )  for  n  /  1  m. 

The  optimum  detector  is  known  to  be  the  likelihood  ratio  detector,  which 
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to  a  fixed  threshold.  Here  f  _  (X )  is  the  c/.ndLtionai  pro’. ability  density  of  the 
received  sa.r  let-  (c  re  a. :  •  ■•i.v’  ■■■.•■  • .  1  i  t  .<■  i  nr;  )  when  siynal  is 

assumed  to  be  present;  r.  r.l’lsr !y  f,.(X)  i.s  :  \r  c  nditurra]  probability  density 
when  signal  is  assumed  u.  he  absent.  Since  X  (-»)  »  X*  (n),  and  since  X(n)  and 
X  (m )  are  independent  ft  r  4  -  n 
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n=l  f..  |X  (n  )"| 

We  assume  now  that  whether  s’ynal  is  present  cr  not  x(t)  is  a  stationary 
GAussian  random  process  with  zero  me;  r.  value.  The  normalized  covariance  matrix 
for  no.is<-  oi.lv  is 
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where  the  superscript  T  ref*  rs  to  n.-pcsii  ten  'u. ;  the  svrtbolv.  /„  means  cnaerible 
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where  ?> (r. '  is  *h«  av<  aae  ••  i,--  a  i 


If  the  signal  is  a  plane  nave,  the  elements  y  (n )  of  Y(n)  are  all  delayed  re¬ 


plicas  of  each  other;  thus  .  2nn  T 

y^(n)  "  c^s(n)  s'-*  '  f  d 


(10) 


where  s(n)  is  the  Fourier  coefficient  of  the  signal  wave  form;  the  c^  are 
weighting  factors  to  take  into  account  that  the  signal  ti’-rength  or  gain  at 
different  hydrophones  may  be  different,  and  is  the  delay  at  the  i^1  hydro¬ 


phone.  The  c^'s  are  conveniently  defined  in  3uch  a  way  that 
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“  V  (n)V1  (n)  (lit) 

P(n)  is  seen  to  be  of  rank  1.  Because  of  the  independence  of  signal  and  noise 
the  covariance  matrix  of  signal  and.  noise  together  is  N(n)  Q(n)  +  S(n)  P(n). 
Therefore  ^ 

fs  L  x(n  )  1  *  ^s+n  (n  )  exT'  |  ~^T(n)  I.N(n)  Q(n )  +  S(n)  P(n)  ]  ~'L  X*{n)|  (13  ) 

and  therefore  the  likelihood  ratio  is 
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Since  P(n)  is  of  rank  1  the  inversion  of  the  second  term  in  the  brackets 
is  easily  accomplished  by  using  the  folic  dr. ••  identity  [  h;  :  For  an  arbitrary 
nonsingular 


matrix  A  and  two  vectors  ’J  and  V 
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Using  this  identity 
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As  is  shown  by  Edelblute  et.  al.  (  3  ]  ,  G  (n)  the  maximum  value  of  the  array 
gain.  Using  this  result  one  finds  tt>«t  the  logarithm  of  the  likelihood  ratio 
is  given  be 
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where  C  -  log  '  ft.  .  (n)  /  0„  (n  ) 
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Since  Q  (n)  «*  Q'(n,  the  quadratic  form  appearing  in  this  expression  can 
be  written  in  the  form: 
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This  implies  that  log  LP.  can  be  obtained  from  a  circuit  of  the  form  shown  in 


Fig.  1  where  the  elements  of  the  fil 
vector  H(n). 


r  ill.  fur  f  *>f  are  the  elements  of  the 
n 


Figure  1  Likelihood  Ratio  Detector 

Except  for  the  scalar  multiplier  this  result  is  the  same  as  that  obtained  by- 
Knapp  [5j  .  The  difference  between  the  result  given  here  (which  is  that  of 
Edelblute,  et.  a'l* )  and  Knapp's  stems  from  the  fact  that  Knapp's  filter 
maximizes  the  output  signal-to-noise  ratio  defined  in  terms  of  a  signal-plus- 
noise  covariance  matrix  Q^,  rather  than  producing  log  LR. 

Ill .  Detection  Index 

In  this  section  it  is  assumed  that  the  signal -to-roise  ratio  is  small; 
i.e.  .i(n)GQ(n)  «N(n)  for  all  frequencies.  In  this  case  one  can  assume 
that  log  LR  is  approximately  a  Gaussian  random  variable  and  that  the  per¬ 
formance  of  the  detector  is  soecified  by 


where  u  n  log  LR,  and  where o  .,(u)  is  the  standard  deviation  of  u  under  i he 

N 

hypothesis  that  there  is  only  noise  at  the  injnit.  The  oouiputation  f^r  <i  is 
straight  forward;  for  details  see  Appendix  A.  The  result  is 
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scalar  is  G  (n).  The  remaining  terrr.s  arc  ?,so  equivalent  t.  s  G  (r  \  gy  j7qs.(29)' 
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and  (19);  thus  or  P(n)  Q  '  (n)  t  «  ii'tn;.  Th  xactcr  of  1/2  appearing  in 
Eq  (28)  does  not  appear  here  because  of  the  complex  notation  that  is  used. 

Thus  Eqn .  (27;  and  (28)  are  identical. 
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IV.  f  .ot  of  directional  Interference 

Suppose  that  the  noise  conponent  of  x^(t,  consists  of  two  parts,  an 
isotropic  part  and  an  interference  part.  It-  is  assumed  that  the  interference 
is  generated  by  R  point  sources.  The  ,oint  source  is  located  at  an 

azimuth  angle  G^,  and  its  spectral  density  is  I  (a);  hence  the  interference 
power  from  the  interference  source  at,  the  frequoncy  o  is  I^n).  The 
desired  target  in  ;  the  azimuth  ingle  ■'  .  «  0,  and  it  is  assumou  tl*ah.  tJi*-*  -army 
is  steered  in  the  target  direction.  The.  inotropic  noise  power  nt  bho 
frequency  <0  is  No(n).  The  is<>  .•nqn'’  o/unjoiuont,  Uk>  tii+o?rftuci»oo  s/aivces, 

and  the  target  signs;  are  all  assumed  to  ho  mutually  i  trip  non  dent,  Gan-ssi  .in  pro¬ 
cesses  with  ze.iv)  Then  the  total  r,->jse  power  density  is  given  by 
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where  (r;  i  s  *  •  ••  lx  .*«•  -*  c.  v-iri  ,nce  •  -<  '!'  the  isotropic  r.oise  com¬ 
ponent  ana  wher-  •  ■  ><  ‘  *!•>!>:•  r.t  ;  •  !••■  '.u:  r.  i  results  iron  one  of  the  inter¬ 

ference  point  sources.  Ry  hr-.  :t  malcvy  *  •  Eqs.  (12),  (13),  and  (Ik) 
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In  most  practical  cases  the  maximum  frequency  processed  *3  such  that  2nWi  >;>  i; 
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which  indicates  that  the  effect  of  the  interference  is  equivalent  to  the  loss 
of  2/ ^  of  a  hydrophone  from  the  array.  This  is  substantially  in  agreement  with 
the  conclusion  of  Ref.  I  l]  where  it  was  concluded  that  the  loss  under  these 
conditions  was  equivalent  to  one  hydrophone;  this  was  arrived  at  by  completely 
neglecting  the  summation  term  in  Eq.  (5h).  Since  '.ho  firure  of  2/^  has  been  derived 
only  for  a  linear  array,  and  since  it  probably  depends  on  the  array  configuration, 
the  figure  of  one  hydrophone  is  probably  a  re--  -enable  e  ,  tin  ate  in  general. 
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Thus  the  result  converges  to  the  case  of  a  •; logic  plane -v.  e  ini-  rf n 
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where  6^  °  ^ —  is  the  angle  half-way  between  tho.  two  interferences.  Then, 
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and  the  two  interferences  are  close  enough  together  so  that  Eq. (68)  holds  if 
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end-fire  axis  of  the  array;  therefore  their  effect  is  always  that  of  a  single 
interference.  This,  however,  is  due  to  the  symmetry  of  the  linear  array  and  does 
not  hold  in  other  cases. 

Mote  further  that  Eq. (72  )  is  a  rather  conservative  limit  since  neither  the 
effect  of  integration  over  frequency  or  over  hydrophone  spacing  has  been  considered. 
Depending  on  the  exact  form,  cf  the  pov;er  spectrum  these  integrations  should  re¬ 
sult  in  increasing  the  value  of  /  -  0  0  j  by  a  factor  of  Ij  or  3  over  that  given 
in  Eq. (72) 
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ference  power,  and  independent  of  the  sracing  of  the  two  interference  sources 
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VII.  More  than  Two  Interferences 


Extension  of  the  results  obtained  so  far  to  mere  interferences  is  diffi¬ 
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from  Eq.(6b)  that  the  off-diagonal  elements  of  G  aie  of  order  /  M.  It  can  be 
shown  in  general  (see  Appendix  B)  that  for  the  purnose  of  approximate  inversion 
an  n-dimensional  matrix  whose  diagonal  elements  are  of  order  k  relative  to  the 
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widely  spaced  poiiT  interference  sources  the  d<  •  -•  i  ity  loss  is  approximately 
equivalent  to  the  loss  of  one  hydrophone  per  interference  source.  The  approxi¬ 
mation  is  good  only  for  R  «  /M. 


VIII.  Effect  of  Distributed  Interference  Source 

A  distributed  interference  source  can  be  represent' d  by  a  large  number  of 
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is  possible  to  obtain  an  analytic  result  which  is  i  r.  :t  ly  applicable  with  .me 
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effectively  the  mean  delay  cf  the  interference  wavefront. 
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i"l  k=i*-l  co  R 
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lo  n 

sin  -2-  (At.  -At.  ) 
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■0«,  u  (  (m)_  ("»))’ 


(BU) 


where,  in  going  from  Eq(83)  to  ( 8U )  we  have  used  the  fact  that 


-(At,  -  A  T .)  =  m(A  i.  -  A  t,  )  =  TfT  -  -  t  ^ 
2  l  k  l  k  i  k 


(83) 


As  before,  the  term  representing  the  loss  of  detectability  is  the  bracketed  term 


in  Eq. (8L).  Except  for  the 


sinx 


term  the  form  of  the  double  summation  is  the 


same  as  that  which  would  be  obtained  for  a  single  interference,  (Ec.  5M  with  a 

mean  delay  t™  at.  the  i^‘  hydrophone,  under  the  condition  Kjh  «  1.  In  fact,  the 

argument  that  the  summation  of  oscillating  terms  tends  to  become  negligible 

c  inx 

applies  here  with  ever  gr  r?r  force,  because  of  the  ~ —  term.  One  can  con¬ 
clude,  therefore  that  for  .i n.tcrfo rence-to-ar.hierr -noise  ratio  snail  enough  to 
satisfy  Eq, (?8),  and  if  the  angle  subtended  by  the  interference  is  relatively 
small,  a  distributed  interference  source  affects  the  performance  in  essentially 
the  same  way  as  a  single  point  interference. 

In  order  to  obtain  an  estimate  of  the  magnitude  of  azimuth  angle  that  can  be 
considered  "small",  consider  a  linear  array  with  M  hydrophones  spaced  d  feet  apart. 
For  such  an  array 


(r) 


where  >:  is  the 
r 


i 

i  —  sin  " 

„  .  t  h 


(86) 


interference  Doint.  Assume  that  the 


inter! 


'•on  c*. 


in  on  = f n 


,■  •  e<  and  is  zero  outside  this 

1.  —  —  C. 


range.  Ihe  cento:  of  the  interference  i3  at  the  angle 


2 


Then,  analogy  with  Sq.(fill  we  expand  .  nVig  • 

sin  0  >  sin  0  <•  1  -  ••  ■  err  •>  (f 

y*  •  ,  i.,  *  M,  1T. 

All  the  other  a  tww  a  le. ■■  •  j.  (81: )  car.  :  :,e:i  be  !  r  .  >  w  m  exact. y  .« 


same  way,  with  t 


•n  over  K  rev;  .  rod  r,-  nr.  Integra*  :or.  '■ver 


final  result-  mi.  N.  ,  into  tin.  form 


F(n)  =  G  -  K_  ii  *  2  v  (M-Jc  j  .os  (— ——  sin  fi  ) 


:<io  d 

sin  I  — — c 


0  /“2  A 
V~  >1 


■*„  Op~0-| 

—  COS  0  (-?— i) 


Except  for  the  teiv.  m  the  summation,  this  is  again  the  expression  th.ru  ire 

would  have  obtained  for  a  single  point  interference  located  at  *  he  angle  a  ,  It 
is  clear  that  the  accuracy  of  this  expression  depends  on  the  accurn  y  of  En.(|;8), 
which,  i.ri  turn  is  a  fairly  good  apr.-roxiration  for  .  -  tt  iess  the  i  about  one 
radian.  Thus  we  conclude  that  an  interference  sourer  spread  over  no  more  than 
one  radar.  at.Tec  tie-  r.ability  essentially  like  a  single  point  interference, 

proviaui  that  KJ:  <c  1. 

Since  the  effr-c  of  interference  for  KM  «  1  is  very  snail,  the 
result  just  ohtaine  i  is  wot  very  .  r.'  cresting  uri  woili  be  desirable  tc  extend 
it  sons  chow  to  the  cone  of  KrM  .»>  1.  »  ni’o  rti.-n  a  t.e]y  this  is  quite  difficult:  in 

j 

fact,  the  only  simple  re  salt  that  ha.  l><  ri  obtained  is  an  extension  of  Eq,  (68) 
to  more  then  2  interference  sources,  if,  jr.  *  he.  case  of  it  interferences, 
it  is  assumed  that  the  mt.  rfer--nce  so  n’s  >re  close  enough  together  so  that 


for  all  frewuencjos  ni  ■ 


■re  close  enough  together  so  that 


then 
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1  or1 
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This  is  easily  inverted  by  use  of  Eq.(l?) 

-l  G 


j  I  +  G  |  y  I  - 


1  *  M  IK 


r  -1 


Also  the  vector  g  defined  in  ,.(h‘  )  becomes; 


(9k) 


(95) 


R  E  o 


ol 


i '  h'  -2  **"  4F 


(96) 


and  therefore  F(n)  of  Eq.(Lv)  becomes 


F(n )  =  M  - 


G  i  l  K 

ol  r-1  r 


D 

1  +  i-i  L  K 


(97) 


Obviously ,  i.f  all  the  Kr  are  °  to  !•"•••.  (07 )  becomes 


F(n) 


Joi  "I 


(98) 


Thus  the  detectability  loss  is  again  equivalent  to  ‘hat  of  i  single  interference 
source  of  strength  K., ,  as  one  could  have  expected.  It  i3  clear  from  the  result 
for  the  case  of  two  point  sources  that  for  a  linear  array  having  M  equally  spaced 
hydrophones  the  maximum  value  of  -•  ,  t  or  whicr.  F.qs .  (97  1  or  (98)  hold  is  given 
by  Eq. (?2 ) ' 

IX,  Compute liona]  f  -  ;  ; 

Since  it  has  net.  eon  ihle  to  oSrt-i.ln  vwatiinful  analytic  result-0  for  cases 
in  which  the  approximations  made  in  the  above  work  are  not  applicable,  F(n)  has 
been  evaluated  on  a  digital  computer  Tor  a  number  of  different  array  and  inter¬ 
ference  pat* and  for  specific  frequencies.  The  results  of  some  of  these 
computations  *»•«  presented  in  Figures  1  through  6.  In  all  comnutations  it  is 
assumed  that  the  array  is  steered  on  target  at  an  angle  0=0  and  that  an  inter¬ 
ference  2X13 ts  at  some  angle  .  The  curves  are  then  plots  of  F(n)  as  is  varied. 
Thus,  if  is  noar  0  the  interference  is  near  the  target  in  azimuth,  and  F(n) 
is  small.  Also,  the  assumption  that  correlation  of  ambient  noise  waveforms  between 
different  hydrophones  is  zero  has  not  been  used;  instead  the  exact  form  of  the 
QQ(n)  matrix  an  given  by  Bryn  |3jwas  used.  As  a  remit  F(n)  f  M  in  the  absence 
of  interference  as  would  be  inferred  from  equations  such  as  (52),  (65),  (75)  or 
(98).  In  fact,  F(n)  <  H  in  all  cases;  however,  this  is  a  coincidence;  it  is 
possible  for  F(n)  >  F  as  1.  shown  by  Bryn  11  .  Tn  ill  casp3  the  interference- 

to-ambient-noise  ratio  is  large. 

Figure  1  shows  the  "i feet  of  a  sing .  pc  ml  interference  with  a  small  circular 
array.  It  shows  that  if  the  interference  directions  differs  by  more  than  about 
U0°  from  the  target  dlrec*  ‘ h'1  "f!-  rr  •;  ’s  essentially  negligible.  It 
must  be  bo  mo  i.-.d,  he-  •  ■  t  rate-}  for  a  single 

a' 


frequency 


■tv  frequencies,  and 


the  integrated  effect  of  all  frequencies  therefr.a,  ha3  the  effect  of  the  loss 
of  one  hydrophone  as  is  predicted  'ey  the  analysis  of  Section  V. 

Figures  (2)  and  (3)  are  similar  to  Figure  1  except  that  the  interference 
co.  a  us  respective!..  ;•*'  *.u.e  ,r.  i  <.j  ,  ,r  jr.t-j,  separated  by  .1  radian.  Since 

*  e  interference  covers  a  larger  a-  •  h  t,  the  effect  on  F(n)  covers  n 

larger  angle ;  however.  ‘  "  .  ,  true  that  for  interference  sources  at  angHee 

for  removed  from  zero  tie  “lie. a  on  FI  is  small.  A  similar  result  is  shewn  in 
Fig.  (U)  which  shows  the  e.fect  of  two  interferences  separated  by  a  large  angle 
(90°).  The  last  two  figures  show  the  effect  of  a  strong  interference  (K^'-POO) 
distributed  over  a  relatively  large  angle  (17°),  Again  the  effect  at  angles  far 
removed  from  the  target  angle  is  small,  but,  as  La  shown  in  Fig. (6),  the  relative 
effect  is  quite  different  at  different  frequencies,  as  has  already  been  pointed 
out. 

The  computations  leading  to  the  results  shown  in  Fig.  1  through  6  were 
quite  time  consuming,  with  computing  times  on  the  order  of  several  minutes  on 
the  Ifl-;  709b  Tor  the  uses  wild  large  numbers  of  interference  points.  For  this 
reason  no  attempt  ras  ,  !e  s  ,  ,  detection  index,  since  this 

would  have  required  summation  of  i-fn  ’  ov..-r  a  large  number  of  frequencies.  The 
computer  results  ’  ref-.i  «•  >t  ill  i.  n •  t  co:  f vely  ansvier  the  question  of  how 

serious  thf  effect  ■  la v..<°  ii  •>  touted  i ;  '  ;  •:-»nces  is.  The  indications  ar9, 
however,  that  the  r-  '.'711  .  .d  under  considerably  wider  con¬ 

ditions  than  those  a,  red  ’  >•.  re  :  reduce  analytical  approximations.  In  fact, 
it  appears  that  loss  of  •!<■:  •  *  si  .  ay  f  -  r  rather  wi  lely  distributed  inter¬ 
ference  is  equivalent  '  .v  •  >  •  :r  ••• 


X.  Conclusions 


The  major  difficulty  in  Lta  ir.i:»g  gene;  al  estimates  of  the  effect  of 
directional  noise  on  the  -*■  *  etifti  1  i ‘7  in  an  vrray  processor  is  that  the 

mathematical  marirt  1  vd  -o'  .  :  ■  •  n  •  r‘  . . .  -e  -re  culte  r'—rlex. 

Results  have  thr  re t  lined  nr-iy  .n  a  r«  ic  number  of  simple 
cases . 

The  general  v..nor  of  1  no 30  io,foi  •,  <  that  if  the  .«ni sotroplc-to-isotropic- 
noise  ratio  is  email  the  efllct  of  a  number  of  local  noise  sources  is  additive; 
that  i3  the  los3  of  detectability  resulting  from  two  noise  sources  of  equal 
strength  is  twioa  that  resulting  from  a  single  source.  For  large  anisotropic- 
to-amblen b-nolsc  ratio  tne  effect  impends  r  whether  the  directional  noise 
sources  are  close  together  or  not.  f  or  ;■  single  point  source  it  has  teen 
shown  previously  and  corroborated  here,  that  the  .loss  in  detectability  is 
approximately  equivalent  to  the  Idss  of  one  hydrophone  from  the  array.  If 
there  are  R  noise  sources,  widely  separated  from  each  other  and  from  the  target 
direction  the  loss  ...  approximately  e«,uiv'.  ier.t  to  ihe  loss  of  ft  hydrophones, 
provided  that  R  where  M  is  ’he  ur  r>«»  of  !  virophcr.es. 

Point  noise  sources  that  are  close  together  affect  the  system  like  a 
single  distributed  noise  source,  and  t ;  * •  indications  are  that  if  such  an 
anisotropy  is  spread  over  a  rclativ-l.v  a:->  Li  or.gle  its  effect  is  essentially 
that  of  a  single  point  noise.  :.fc.  tun  o-ety  tr. , a  has  not  been  conclusively 
demonstrated,  even  t\y  use  oh  y,  •  t«-r,  n J  or  iy  a  rather  conservative 

estimate  of  azimuth  angle  that  can  be  considered  to  be  "small"  has  been 


obtained. 


Apppn  iix  A  Derivation  of  the  Detection  Index 
"ii.  i  .ention  index  d  is  given  by 
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Sj  mill  avly,  and  using  the  fact  that 
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To  find  n  (n )  it  is  necessar;.  t  r  t  :::d  '-a  '  given  by 
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Let  w(n)  -  H1  (n )  X(n),  then  w(n)  is  a  Gaussian  -andom  variable  sine'  %  (ji  } 
is.  In  terms  of  w^n ) 
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This  expansion  is  permissible  because  w(n)  as  Gaussian  !  ill.  The  first  term  in 
the  expansion  is  simply  the  square  of  the  mean  Nu)  ^  the  second  term  vanishes  [  5  I, 
and  in  the  last  term  all  terms  for  which  n  /  m  vanish  because  components  at  diffe¬ 
rent  frequencies  are  assumed  to  be  independent.  Hence 
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as  in  Eq.  (A -7  ). 
Thus,  finally 
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Fox1  small  signal-to-noise  ratio,  such  that, 


K(n  )  £  S(n  )  /  i.'^  (r. ) 


?hen 


r  -x 


{ ,  ) 

i  i  . 


.  (n  )A'.  (n  ;  <  '  j 


(n . 


( A-1 1 


(A-ll: ) 


(A -15  ) 


I A -16) 


Appendix  B 


Approximate  Inversion  of  a  Matrix  whose  Diagonal  terms 
are  Large  Relative  to  the  Off  -  Diagonal  Terms, 

Let  the  nxn  nonsingular  matrix  A  be  given  by 

A  =  D  :  B  (B-l ) 

where  D  is  diagonal  and  B  ii  a  matrix  with  zero  diagonal  elements.  It  is  assumed 
that  all  the  non-diagonal  elements  of  B  are  of  about  the  same  order  of  magnitude, 
and  that  the  elements  of  D  are  of  about  K  times  that  magnitude,  with  K  >  >  1. 

The  inverse  of  A  is  given  by 


A"1  «  (D  +  B)*"1  »  D-1  (I  +  BD~1)"'1  »  D_1 


1  -  BD-1  + ( BD-1  , 


(B-2 ) 


Since  the  elements  of  D  are  of  order  K  relative  to  B  the  elements  of  B  D-  are  of 
order  1/K  relative  to  unity. 

It  can  be  shown  [8.]  that  a  sufficient  condition  for  convergence  of  Eq,  (B-2) 
is  n 


E  b  <  1  i  -  1 . n  (B— 3 ) 

3-1 1  -11 

where  b^.  are  the  elements  of  BD  \  Assuming  all  of  these  elements  to  be  of  about 
the  same  order  of  magnitude,  condition  (3-3/  can  be  expressed  in  the  approximate 

form  nb  <1  (B-U) 

o 

where  b^  is  a  representative  element  of  BD_\  This  element  is  of  order  1/K; 
therefore  convergence  requires  n/K  <1  (B-5) 

The  convergence  will  clearly  be  more  rapid  if  this  inequality  is  sharper;  hence 
one  can  approximately  n6glect  the  matrix  B  in  the  inversion  of  A_  if  n/K  «  1. 
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ABSTRACT 


In  this  report  some  studies  have  been  carried  our  for  a  class  of 
adaptive  filters  consisting  of  tapped  delay  lines  and  adjustable  gains. 
The  method  of  stochastic  approximation  and  mean  square  error  criterion 
are  employed  to  adjust  the  gains  automatically .  it  is  shown  that  it  Is 
not  necessary  that  the  d>  ;  :  I  g  1  '  ■->  train  the  •  »  r  r 

function  is  available.  Hither  signal  r  noise  corn,  let  ion  functions  will 
suffice  to  generate  the  error  gradient.  Problems  basic  to  all  adaptive 
processes  such  as  the  conditions  for  convergence,  rate  of  convergence, 
effect  of  misadjustment ,  effect  of  time-varying  parameters,  and  the  re¬ 
lationship  between  mean  square  error  and  the  number  of  delay  elements  are 
answered  with  explicit  expressions. 
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I.  INTRODUCTION 


1.1  Optimum  Linear  Filters 

The  problem  of  designing  a  dr-'1. h  n  i ;  --  »  itc  noise  or  to  predict 

the  future  behavior  of  an  incoming  signal  has  been  considered  by  Norbert 
Wiener"  more  than  twenty  years  ago.  Ibis  kind  of  device  has  been  termed 
as  "filter1'  in  general.  Consider  a  linear  filter  shown  in  Figure  1,  where 
the  input  x(tj  is  a  combination  of  the  usciui  signal  s(t  ■  and  noise  n(t). 
/issuming  that  n(t)  is  additive  to  and  statistically  independent  of  s(t), 
we  have 


and 


x(t. )  «*  s  (t )  -•  n(t ) 


(l.D 


s  (t )  n(t )  "  0  if  s  (t )  “  0,  n(t )  =■  0 


y(t) 


(1.2) 


Figure  ].  A  linear  filter 

The  output  y(t )  of  the  filter  is  to  approximate  a  desired  luncticn 
d(t)  which  is  related  to  the  signal  s(t).  The  performance  criterion  to 
be  minimized  is  the  mean  square  error 

ec(t)  "  (d(t )  -  y(t)j2  (1.3) 

This  is  the  classical  problem  of  Wiener,  the  analytic  solution 

(for  the  impulse  response  of  a  realizable  filter)  is  known  to  be  the  solu- 

? 

tion  of  the  V.'iener-Hopf  integral  equation 


Sxd<T> 


h(t )  )  dt 


with  the  solution* 


„  ,  ,  ^xd^ 
0  cpxx(J>) 


(1.U) 


(1.5) 


and  the  minimum  mean  square  error 
—  +ft> 


2 
e  . 
min 


d2(t)  -  "2(t) 


(1.6) 


In  the  above  equations  R  (T)  and  (w)  are  the  cross  correla- 

xy  i  xy 

tion  function  and  spectral  density  functions  between  x(t)  and  y(t). 
y  (t)  is  the  output  of  the  optimum  filter.  The  results  are  valid  for 


stationary  signals. 


.3 


Kalman  and  ducyJ  have  presented  a  new  method  to  design  optimal 
linear  filters  for  nonstationary  signals.  They  considered  the  model  for 
the  signal  process  as 


ax 


it 


Fx  t  Ciu 


where  u(t)  is  white  noise,  i.e., 


i  (t )  u  (t,)  =  N(( ft-T ) 


(1.7) 


(1.8) 


The  observed  signal  is  assumed  as 


=  HX  +  V 


(1.9 ) 


where  H  -  (1,  0)  and  v  is  white  noise  of  spectral  density  fj)  . 


*This  type  of  filters  mav  not  he  phvsi-'nlly  realizable.  Further  discussions 
can  be  found  in  Wiener'* 


I)-:i 


a. lo) 


The  optimum  filtering  problem  consists  in  determining 
x(t)  =>  E  (  x(t)  |  7.(1)] 

A 

where  x(t),  the  conditional  t xpectation  of  x(t)  given  the  observation 
a(T)  in  the  interval  (0,  t ),  is  the  ••  ir.ir-:.n  variance  unbia  e-i  estimator  of 
x(t). 


The  optimum  sysu  •  j  .  described  ,uc  in  fly  by  the  following  four 


equations  : 


with 


and 


dx  a  T  ,  _1 

^  -  Ex  +  PH  q)  (z  -  Hx) 

x(o )  =  0 

dn  T  T  j.  -1 

•^r  =  KP  +  PF  -  PH  cp  x  HP  + 

P  (0 )  =  E  x(o  )  (o  ) 


0  l 
0  N  ) 


(1.11  ) 


For  time  varying  systems,  Kalman -Buoy  filters  can  provide  much 
better  performance  than  Wiener  filters.  From  (1.0)  nnd(l.ll)  it 

is  seen  that  the  statistical  properties  of  both  signal  and  noise  should 
be  known  in  order  to  design  either  type  of  the  two  filters. 

Thus,  if  the  a  priori  information  is  not  known  completely,  opti¬ 
mal  performance  of  the  filters  cannot  be  expected.  In  .an  attempt  to  re¬ 
cover  sane  of  the  missed  a  priori  infona.it. i  u  /  valuation  of  the  actual 
performance  of  the  operating  system,  tl  *•  r  ncept  of  adaptation  has  been 
developed  and  accepted  .as  one  of  the  p>  ...  t ■  1  •  so".  W  ions  .ny  such 
problems. 

1.2  Adaptive  systems  •«:. !  .  tc.1  1  . 

A  system  is  a  da  .  ti  ii  ere  i  tee :  .•<  <  ■ . -i.e  hi  l:ii  m  the 


feedback  mop  where  she  ooservable  output  is  compared  vdth  the  desired 
output  so  as  to  adjust  the  syster  'or  better  performance  ’  In  other 
words,  an  adaptive  system  is  ewe  which  is  provided  vdth  a  means  of  con¬ 
tinuously  moni tcrir.fr  ids  own  performance  according  to  a  given  performance 
index,  and  also  a  means  of  adjusting  its  own  parameters  by  closed  loop 
action  so  as  to  cp'.imiae  its  own  operation.  Relaxation  method  and  method 
of  steepest  descent  r  ascent)  are  two  of  the  most  ccrrmor.l'r-used  adap¬ 
tive  techniques. 

The  relaxation  method  involves  making  a  change  in  the  value  of  only 
one  oi  the  controller  parameters  and  then  re-evaluating  the  performance 
measure.  T  the  performance  has  been  improved,  a  second  change  in  the 
same  dirtmticn  is  ..ado;  otherwise,  the  first  change  is  retracted  and  a 
change  in  the  opposite  direction  is  made.  This  process  is  continued  until 
no  further  improvement  in  the  performance  measure  can  be  accomplished 
by  adjuc.  ...  that  particular  parameter;  whereupon  the  sane  process  is  re¬ 
peated  for  'Men  rf  i.he  remaining  controller  parameters.  After  several 
iterations  through  the  entire  procedure,  the  controller  parameters  tend 
toward  that  set,  of  values  which  yields  the  optimum  performance  measure. 

Phe  methods  of  steepest  descent  (or  ascent),  referred  as  gradient 
techniques  are  operated  in  a  manner  similar  to  the  relaxation  method, 
with  the  notable  exception  that  all  parameters  are  adjusted  simultaneously 
rather  than  sequentially.  This  is  done  by  measuring  the  partial  deriva¬ 
tive  of  the  oerfcrmance  measure  with  respect  to  each  of  the  controller 
parameters  and  ti.-.-i  adjusting  all  the  parameters  in  such  a  way  that  the 
net  effect  is  tim*  .  vrgest  possible  improvement  in  the  performance  measure. 
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A  number  of  techniques  have  been  developed  for  determining  the  partial 
derivatives. 


The  most  straightforward  metre  d  i  -  to  perturb  each  of  the  para¬ 
meters  sequentially  and  measure  the  derivatives  directly.  This  procedure, 
however,  offers  little  advantage  over  the  relaxation  metnod.  A  second 
technique  is  to  perturb  the  :•  camelem  simultaneous  ty  ir.  such  a  manner 
that  the  effect  of  the  perturbation  of  each  parameter  on  the  performance 
measure  will  be  distinguishat . ;  from  the  effects  of  the  perturbations  of 
all  the  other  parameters,  bays  in  which  this  may  be  done  include  per¬ 
turbation  by  independent  random  noise,  distinguishing  the  individual  ef¬ 
fects  by  correlation  detection';  or  perturbation  by  frequency-separated 

0 

sinusoide,  distinguishing  the  effects  by  narrow-band  detection  .  Gradient 
techniques  can  be  consider'd  as  the  special  case  of  the  more  general 
method  of  stochastic  approx  mat  ion,  by  which  either  deterministic,  or 
random  problems  can  be  solved  with  ease. 

1„3  Adaptive  Filters  and  State  of  the  Art 

iidapti-n  filters  have  been  investigated  by  a  number  of  research¬ 
ers^  1  Their  a e  the  Is  differ  cniofly  in  t.he  ways  of  implementation, 
but  all  are  designed  with  tn-  same  purpose  in  mind  -  to  extremize  the 
performance  index  by  gradually  adjusting  the  system,  parameters.  One  of 
the  simplest  implementations  in  this  area  is  the  use  of  tapped  delay  lines 
which  can  be  constructed  easily  with  shift  registers  in  dgital  computers. 
Weaver  ^  considered  tapped  delay  line  filters,  but  his  adaptive  scheme 
was  rather  ineffective  and  required  the  ml:  ion  of  many  simultaneous 
equations.  Narendra  and  the  author  ^  u^d  -'"lav  lines  to  identify  tna 
characteristics  of  some  unkr^wn  ?  -rl  i  ••  !  tic  systems.  Although  the 


feasibility  of  the  above-mentioned  methods  was  .indicated  by  computer  simu¬ 
lations,  some  figures  of  merit  to  judge  these  schemes,  such  as  the  rate  of 
convergence  of  the  parameters  to  the  optimum,  remain  untackled.  Widrow'^*  ^ 
has  attempted  to  solve  these  problems  by  defining  some  adaptive  constants 
and  misadjustment  formulas.  His  system  consists  of  delay  lines  and  ad¬ 
justable  gains,  and  has  been  acclaimed  to  perform  nearly  as  the  Kalman- 
Bucv  filter  when  perfect  a  priori  information  is  available.  Under  cir¬ 
cumstances  in  which  the  a  priori  information  is  not  perfectly  known,  it 
is  quite  possible  that,  the  performance  of  such  an  adaptive  fixter  could 
exceed  that  of  either  a  Wiener  or  a  Kalman-Bucy  filter.  However,  Widrow's 
system  requires  the  availability  of  a  desired  signal  to  generate  the  real 
time  error  function.  Convergence  proof  of  his  LMS  adaptation  algorithm 
(least-mean-square-error  algorithm )  was  not  given,  and  the  development 
of  the  rate  of  adaptation  was  not  mathematically  rigorous.  Moreover, 
how  to  make  a  time-varying  system  adaptive  has  not  been  considered. 
l.U  Outlines  of  the  Report, 

A.  Con' ider  a  random  function  Q(x|c)  where  x  =  f  x^,  Xj>  ...,  xfi  ] 


J 


is  a  vector  o  stationary  random  process  with  distrioution  I'  (x).  In  an 
attempt  to  m.nimize  the  criterion 


1(c) 


[Q(x(c)  ) 


it  is  natural  to  set  the  gradient  of  1(c)  to  zero, 

7  I(£'  =  Ex|  Vc  Q(xi£)| 0 


(1.12) 


(1.13) 


Since  P(x)  is  generally  unknown,  an  algorithm  derived  from  the  method 
of  stochastic  approximation  to  obtain  c',  the  optimum  value  of  c,  is 
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O.lij) 


c  .  .  =  c  .  -  y  7  r'  (x  .( c . ) 
-j+i  -.1  }  j  'c  '-J  -J 


Algorithm  converges  with  probability  or 


p[  I/m  (£j  ~  £*)  =  0  |  -  1 

^ /vn  J 


1  J  00 

as  well  as  in  mean  square 


lim  E  j  j  |  £  j  “  £ 
j  -too  ’ 

under  tha  following  conditions 


w  CP 

A.  /."vjj".  =  0,  =■  5”  Y  .  <  ",  ^r. 

u  3  ’  J-l  ^  J  y  .1  ’  .1 

J:l 

B.  inf  E  |  (c  -  ctt)T  Q(x|£)|  >  0 

e  <  II  *rl\\  <  1.  *  C 

C.  E  jvj  0(x|c)yc  Q  (x  |  c  )j  <  d (c  ’ 1  o'  »• 


T  , 

ice) 


'  r  all  c  and  d  >  0 


(1.15) 


(1.16) 


(1.17) 


(1.1  O' 


(1,19; 


B.  For  the  tapped  delay  line  filter  under  study,  the  transfer 


function  of  the  filter  is  represented  by 


HM  ■  E  -k  .-•*** 


(  In  1  ) 


and  its  impulse  response  is 


h(t)“  l  '^'"V 


0.21) 


If  we  attempt  to  minimize  the  mean,  square  error 


e2  -  e|  (d(t)  -  ,(t)  )2j  -  E  j(d(t) 


-  £  1 


k=0 


2 

(i:22) 


the  optimum  values  of  c  is  obtained  as 

£*  “  ?-)  _1  hr, 

where 


Rq.  =  E 


with 


(1.23) 


,  R .  =■  E 

/  J  \ 

d1o 

r(o 

’  “drj 

Tj^t)  =x(T-Tk) 


ic  is  seen  that  in  (1.23)  the  values  of  'c*  cannot  be  determined 
unless  we  have  full  knowledge  about  both  signal  and  noise  correlations. 

Substituting  .  (1,23)  into  (1.22)  we  can  obtain  the  ex¬ 
pression  for  the  minimum  mean  square  error  as 


2  a2 ,,  v  ,  T  «■ 

e  .  =  d  (t )  -  ri.  .  c 

min  dh 


,2, 


\  '  Ri  a<t 


1 


“  d2(t)  -  z2(t) 


(lo2U) 


where  zo(t)  is  the  output  of  the  optimum  filter. 

The  mean  square  error  at  ar.y  time  for  arbitrary  values  of 

c  is 


e2(t)  =  e2.  +  (c  -  cY  R  (c  -  c") 

min  -  -  -  q  -  - 

and  the  effect  of  non-optimum  setting  is  bounded  by 


(1.25) 


e  ( t )  -  e  .  <  (N+l )  max 

min  -  , . 

all  1 


1  •»  1 
c.  -  c. 

max 

n  -n  ■ 

1  1 

1 

t  17  j 

(1,26) 
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A  relationship  between  the  Minimum  mean  square  error  and  the 


number  of  delay  elements  is 

e2.  -  R  (o)  -  R  T  Rn  _1  R  (1.27) 

The  last  term  in  the  right-hand  side  of  (1.27)  is  a  functional 

of  N,  the  number  of  delay  elements  and  the  correlation  functions.  For 

2 

any  known  forms  of  R  (T)  and  R  (7").  a  plot  of  e  .  versus  N  can  be  con- 
n  s '  '  ’  ^  min  _ 

2 

structed.  It  is  anticipated  that  the  larger  N  is,  the  smaller 
will  be. 

C.  Under  various  practical  situations  we  may  not  have  perfect  in¬ 
formation  about  both  R  (T)  and  R  (7").  Techniques  of  adaptation  can  be 
employed  to  estimate  the  incomplete  a  priori  information  and  to  make  the 
filter  adaptive  to  changing  operating  conditions. 

From  (1,22)  the  error  gradient  without  averaging  operation 
is  obtained  as 

Vc  Q(£l£)  15  “2  (  d(t)  -  z(t)  )fj(t) 

so  that  the  desired  adaptive  algorithm  is 

+  (1>28) 

The  above  algorithm  converges  if  1.  Q(o)  is  strictly  convex, 

32Q 

2.  exists  and  is  uniformly  bounded,  3„  s(t)  and  n(t)  are  uniformly 

bounded. 

In  v1.23)  the  desired  signal  d(t)  is  used  to  generate  the 
error  function.  This  assumption  is  not  so  practical  when  dealing  with 
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detection  problems.  If  only  the  noise  correlation  function  is  known,  we  can 
change  Eq.  (1.28)  to 

wnere  H?  =  (Rfl(o),  ...,  Rs  (Tj) 

On  I  he  other  hand,  if  only  tho  signal  correlation  function  is  known,  we  have 


Wi  (1-30) 

where  4  -  (k.(o),  fis  (T  Jf 

E.  The  other  problem  that  we  have  to  consider  is  the  rate  of  adaptation, 
or  the  rate  of  convergence.  VJe  want  to  estimate  how  fast  the  gains  approach 
their  optimum  values.  Defining 

EJ(Vj)  =  *h)  -  P.  _1  /\  P  (1.51) 

W  ■  =  P  c  ,  jj1  =  P  r|  }  (1.32 ) 

and  yP  =  J ( jTTJx-  P°r  P^  ComPonen't'  of  > 

P 

we  can  express  the  component  of  W  at  any  time  during  the  adaptation 

period  as  a  function  of  the  initial  oholcc'  and  the  optimum  values  of  E,  i.e., 


vrr  (w.  -  v*)  +  w* 
l  +  l  1 


(1.33) 


Using  .  (1.2b)  we  obtain  the  difference  between  e  (t )  at  any  time 


and  e  .  as 
mm 


2  2 

e  .  -  e  . 

1  mm 


..  wi  n  „ 

4  ).  <4  -  a.)2 

pt0  P  J  3 


(1.3  U) 


( ,S’+1)  emu  i  n 
J 


.2  j  -  C")T  ^  (=1  £*> 
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(1.38) 


t  ■*  p  t 


then  the  method  of  two  time  scales  can  be  used  to  modify  the  algorithms 
obtained  previously.  The  mean  square  error  is  changed  to 


e2(t)  ^  E  j(d(t)  -  Z  c±  ,(t)j 


/  N  N  3c .  (t ) 

+  2pT  £  fd(t)  -  3"  c.  ,  (t)]  T  n.  (t)  — £«— 

av  )  '  >0  1  1  }uo  fi  ot 


(1.39) 


where  T  in  the  average  delay  time.  Algorithm  (1,28)  is  modified  to  the 
form 

2m  ■  23  *  %  h  W  *  2rj  P  3j  $  (1-h0> 


where 

3c.  (t) 

3? 

The  minimum  mean  square  error  for  this  case  is 


2 

e  < 
:  .  n  - 


0  Kill 


P’av!  & 


2  2 
Tav I 


§  T  ^  §  |(i.ia) 


where  e  .  is  the  minimum  mean  square  error  of  the  time-invariant 
o  mm 

filters. 

1,5  Research  Work  ir  rrogi  is 

So  far  preliminary  re  .,ts  have  been  obtained  for  a  single  input 
simgle  output  filter  under  very  general  situations.  Practical  examples 
for  different  types  of  s'  -nal.  \  ise  and  ways  of  parameter  variation  are 
being  worked  out.  Tn  on  ur  to  •  rif"  the  results;  digital  comp  iter  simu¬ 


lation  will  be  conducted. 


where  is  the  p^*1  component  of  A  and  is  the  p*1*1  component  of  w^. 

E.  So  far  w  e  have  assumed  that  the  whole  system  is  time-invariant 
and  the  output  signals  are  stationary  at  least  in  the  wide  sense.  How¬ 
ever,  som-’i  times  lor  one  reanon  or  another,  system  parameters  or  signal 
properties  may  change  slowly.  One  instance  of  this  situation  is  the 
fluctuation  of  power  levels.  Same  schemes  to  adjust  the  gains  under  this 
case  would  be  highly  desirable.  First  consider  the  quasi-stationary 
case  where  only  slow  time  variable  is  involved.  If  a  time  function  x(t) 
is  delayed  by  an  amount  of  T  and  multiplied  by  c  such  that 

y(t)  «  cx(t-T) 
or 

y(t+T)  =  cx(t) 

which  can  be  written  as 

Ly(t )  *>  cx  (t )  (1.35) 

L  is  a  linear  differentiation  operator 

L  =  e*3'1'  =  II  p*  and  p  »  -4r  (l»36) 

io0  ' 

Let  x(t)  =  e‘^  and  y(t)  «  z(t,\)e^,  then  z(t,X)  can  be  obtained 
as  zQ(t,X)  =>  ce~^ 
if  c  is  a  constant,  and 

z  (V-)  “  -TE 
e" 

if  c  is  time-varying.  The  effect  of  time-varying  parameter  is  observed. 

During  the  training  period  the  system  is  operated  in  real  (fast) 
time  t.  If  at  the  same  fume  some  parameters  ox  ,  clanging  slovly  in  slow 

A 

time  t  such  that 


.  "  1  A<c> 

(  O  -  a  JT  — j—  ) 

i=l  •  dt 


(1.37) 


D-n 


A  very  interesting  and  practical  application  of  adaptive  filters 
is  the  sonar  detecting  system  consisting  of  many  hydrophones  steered  or 
not  steered  on  target.  The  output  of  each  hydrophone  passes  through  an 
adaptive  filter,  and  the  sum  of  the  filter  outputs  is  squared  and  averaged 
to  indicate  the  presence  or  absence  of  a  target.  When  the  input  signal 
to  noise  ratio  is  small  and  the  levels  of  signal  and  noise  are  the  same 
at  all  hydrophones,  the  whole  system  can  be  adjusted  to  form  a  likelihood 
ratio  detector.  Some  features  of  this  detector  will  be  studied  with  em¬ 
phasis  on  adaptive  schemes,  rate  of  convergence  and  detectability  for 
stationary  and  time-varying  processes.  Performance  analysis  will  be 
carried  out  with  analytic  and  numerical  examples. 


II.  METHODS  OF  STOCHASTIC  APPROXIMATION 


1.  Historical  Developments 

The  methods  of  stochastic  approximation  were  originally  developed  by 
Robbins  and  Monro  in  1951  Their  purpose  was  to  find  the  root  of  a 
noisy  function.  The  term  "stochastic"  refers  to  the  random  character  of 
the  experimental  errors,  while  the  term  "approximation"  refers  to  the  con¬ 
tinued  use  of  past  measurements  to  estimate  the  approximate  position  of 

l<t 

the  goal.  Kiefer  and  Wolfowitz  adapted  the  idea  of  stochastic  approxima¬ 
tion  to  the  problem  of  finding  the  maximum  of  a  unimodal  function  obscured 

by  noise.  Blum‘S  used  the  gradient  method  to  extend  the  above  techniques 

17 

to  multi-dimensional  case.  Later  on  Dvorestzlcy  greatly  generalized  and 

10 

unified  the  whole  theory  and  Kesten  derived  some  formulas  to  speed  up  the 
rate  of  convergence  in  terms  of  the  number  of  changes  in  sign  before  a 
certain  step,, 

19 

2.  Basic  Considerations 

Stochastic  approximation,  much  like  ordinary  successive  approximation 
in  the  absence  of  experimental  error,  involves  two  basic  considerations  - 
first  choosing  a  promising  direction  in  which  to  search  and  selecting  the 
distance  to  travel  in  that  direction.  Picking  a  search  direction  is  no 
more  difficult  for  stochastic  tka  n  for  deterministic  approximation^  for 
one  simply  behaves  as  if  he  b.-Mieved  the  experimental  results,  ignoring  en¬ 
tirely  the  possibility  of  err  m.  This  means  of  course  that  the  experimenter 
will  move  array  from  his  goal  /./-never  he  is  misled  by  the  vacc.ries  of  chance 
error.  It  will  be  seen  that,  cveh  temporary  sel  -backr  b  net  prevent 


I)— 1-1 


ultimate  convergence  if  the  step  sizes  are  chosen  properly. 

In  both  stochastic  and  deterministic  schemes,  the  corrections  are  made 
progressively  small  as  the  search  proceeds  so  that  the  process  will  eventually 
converge.  To  make  this  convergence  rapid,  one  would  like  to  shrink  the  step 
size  as  sneedily  as  possible.  The  main  difference  between  stochastic  and 
deterministic  procedures  is  in  fact  the  Speed  with  which  the  steps  can 
be  shortened.  When  noise  is  totally  absent  one  can  reduce  the  steps  very 
rapidly,  but  when  there  is  danger  of  an  occasional  jump  in  the  wrong  di¬ 
rection,  shortening  the  steps  too  rapidly  could  make  it  impossible  to 
erase  the  long-run  effects  of  a  mistake.  In  the  latter  case  the  process 
would  still  converge,  but  to  the  wrong  value. 

20 

3.  The  Methods 

Many  problems  in  modern  cybernetical  systems  design  can  be  reduced  to 

that  of  finding  the  extrema  of  functions  of  several  variables 

I  “  Q(c1?  c2,  ...,  cr)  -  Q(c  ) 

(  „  (2.1) 

where  c  «■  )  c2,  ...  cp  j. 

'  ’  ; 

Denoting  the  optimal  values  of  £  by  c  and  assuming  that  the  ex¬ 
tremum  of  interest  to  us  is  a  minimum,  we  can  obtain  the  solution  of 
c  =  c*  by  setting  the  gradient,  of  Q(c)  equal  to  2  j  i.e., 

V Q (5  '  '0  (2.2) 


D-  Id 


Generally  a  closed-form  solution  cannot  he  obtained  for  (2.2^30 
iteration  methods  are  required,  especially  the  gradient  method. 

The  gradient  method  relates  the  coordinates  of  a  given  point  with  the 
coordinates  of  the  preceding  point  and  the  gradient  V  Q(c).  The  algorithm 

■ft 

for  determining  c '  can  be  written  in  the  form 

■  -i  (2-3> 

Here  ^  determine  3  the  pitch  of  the  algorithm  and  generally  deponds 
J 

on  the  index  of  the  step  and  the  function  itself. 

When  Q(c)  is  not  given  analytically  or  is  not  differentiable,  the 
gradient  VQ(c)  can  be  approximately  determined  with  the  formula 


Q+(c,  a)  -  Q_(c,  a) 


2a 


where 


Q+(c,  a)  =|Q(c  t  ae^),  ...,  Q(c  t  aen)  j 


and  e.  denotes  tne  base  vectors 

i 


£i 


1,  0,  •  •  < )  o") ,  !o,  0,  ....  1 


(2.U) 


(2.5) 


The  corresponding  algorithm  is  then 

r  Q-  a  )  -  Q  (c  ,  a  )  o 

c.  ,  =  c.  -r.  \  -  - J— ---  J - i-  1  (2.6) 

-3  h]  2a.  ; 

In  the  above  we  assured  th  Q(c)  is  a  detc  '.inistic  fraction.  If 

we  consider  n  random  function  (,ifx|c),  where  x  -  j  x_ ,  ,  x  1  is 

-  f  i  c.  n  j 

a  vector  of  stationary  random  icesses  with  dis; , ibuti:  l  p  (x),  it  is 
natural  to  attempt;  to  find  the  •  xtrema  of  she  mat.  mail.  al  expectation  s 


D- 1 (J 


(2.7) 


I(.£)  m  j  Q(x|c)  P(x)  dx  °  Ex|  Q(x|c)|. 

'  x  ' 

The  condition  for  determining  the  optimal  -value  c  ■=*  c  is  of  the  form 
V  1(c)  =  e|\7cQ(x|  c)  |  =  0  (2.8) 

We  can  apply  the  algorithms  (2,3)  and  (2.6)  to  (2.8)  and  functional 
(2.7)  only  when  the  a  priori  distribution  P(x)  is  known  and,  consequently, 
the  mathematical  expectation  (2.7)  can  be  determined  beforehand.  Fre¬ 
quently,  however,  the  probability  density  function  p(x)  is  unknown. 
Nonetheless,  the  optimal  vector  c  =  c*  can  still  be  determined  by 
applying  the  gradient  method  using  SJ  Q(x<c)  instead  of  E>\7  Q(x(c )(. 

c  -  -  rc  -  -  j 

This  is  the  advantage  of  using  the  method  of  stochastic  approximat ion. 

V/ith  this  method  the  algorithms  for  determining  c  =■  c*  can  be  written 
in  the  form 


c  .  =  c  .  -  Y  ,\7  Q(x  .1  c  . ) 

-J+1  -J  J  J  v  c  -J  -3 


(2.0) 


if  Q(x(c)  is  analytic  and  differentiable,  and 


if  V  Q(x|c)  doc;:  not  exist, 

0  —  — 

Algorithm  (2,9)  is  a  multivariate  form  of  the  .lobbies -Mom o  procedure, 
while  algorithm  (2.10)  is  a  mui  ( v»  jriate  form  of  <-  Ki of er-Wolf owi tz 
scheme.  The  analogy  between  de'  orministi <.  and  stochastic  elgc.'  Ithms  is 
apparent.  It  should  be  er.pbas  -  :,cd  however,  that  Lc  algorithms  deal 

with  stationary  random  variables  which  may  c  ntai  ranacn  noise  in  a  ,Ji- 
tion  to  the  useful  signal.  The  ccnvet pence  properties  of  the  above  al¬ 
gorithms  will  be  considered  t  he  nert  --ct  -•n. 


!>-  i 


U.  Convergence  Property  es 

In  this  section  the  conditions  under  which  the  above-mentioned  al¬ 
gorithms  converge  will  be  described.  Since  mean  square  error  is  used  in 

this  report  as  the  only  performance  criterion.  Q(x|£)  is  analytic  and  differ¬ 
entiable,  and  we  therefore  need  to  consider  only  algorithm  (?.9). 


Let  c'  satisfy  the  equation 

E|VC  Q(x|c)j  -  0  (2.11) 

E|\7C  Q(i(o)j  is  a  set  of  real  measurable  functions  of  real  variables  c 
such  that 


E 


fVc  Q<*l£) 


0)1 


! 


>  0  for  c  >  c 
<  0  for  c  <  Y 
a  0  for  c  =  c 


(2.12) 


*>{■  -ft 

where  c  >  c  means  c.  >  c.  for  all  i. 

—  <  -  l  <  i 

Theorem:  Let'J'p'^jJ  •••  1:163  sequence  of  positive  numbers  such  that 


(Al) 

l.'w  Y.  =  o 

0 

oo 

(2.13a) 

(A2) 

j=i  - 

CD 

(2.13b) 

(A3) 

ir  <» 

3=1  J 

(2.13c) 

V° 

(2.13d) 

Let  the  following  conditions  be  satisfied 


fB) 


inf 


E(  <c  -c*)Ttfc  Q(x|c)J  >0 


(2,  llj) 


e<  c  -  c  <  -- 


e  >  0 


D-ls 


(C) 


(2.15) 


S|vc  r  Q(il£)Vc  Q(x'£)|  S  d(c^Tc  *fcTc  ) 

for  all  c  in  a  bounded  set 
and  d  >  0 


Then  the 

*  21 

c  . 

Proof: 


sequence  c.  defined  by  (2.9)  converges  with  probability  one 
3 


Subtracting  both  sides  of  Eq.  (2.9)  by  c  we  have 


a 

c 


where,  for  simplicity,  0  =  0  (>c  I  c ) 
Squaring  Eq.  (2.16) 


(c  -  c*)^  (c  .  -c+  )  =  (c  .  -  c  )T  (c.  -  c  '  ) 
-3+1  ~  1  -j+1  -  -3  -  -3  - 

-2Y  (c  -  c*)TtfQ 

J  J 

+  (T2  V  tqVq 

0 


and  taking  the  conditional  mathematical  expectation  for  given  Cg, 

c we  obtain 


From  condition  (q),  (2-17)  becomes 


•  d  (£Vr 


to 


(2.16) 


(2.17) 


(2.18) 


i  >-  1  .■ 


Using  condition  (B),  (2.18)  is  reduced  to 

*"21-  cl 


E  /  c  .  -  -  c 
-J+l  ~ 


£l* 


‘i  i 


<  ||cj  -£*||2  (1  +y2.  d)+  2^  dcT  c* 


Let  Z  .  =  c.-c*  r  ]T  (1+yf  d ) 

-J  -J  -  1  k 


’h'2^  (l+)f2 
k- j 

+  I  2dTk  ^  £*  ^  (1  d) 

k= j  m=k+l 

Tt>e"  -i+l  ’  IUl+l  -  °"!!2  K  4) 


k°  j+l 


+  l!  2dV2  cT  c"f  7T  (1  +t!  d) 


’k-  -  - 
k=j+l  m“k+l 


(2.18a) 


(2.19) 


(2.20) 


Taking  the  conditional  mathematical  expectation  for  given  c^;  c^,  . 
c . ,  we  have 

-y 

E  I  >i  l-i’  *eI' 


£j+i  “  £  |l  k^+1  (1+^k  d) 


k=j+l  iV  m°k+l 


(i^d) 


*  Z  2<^£T£*  K  Odfljjd) 

k=3+l  m=k+l 


Z  . 


°r  E  {~j+l|£l,  Hi  j 


(2.21) 


D-20 


Next  taking  the  conditional  mathematical  expectation  for  given  Zp  . 
Zj  on  both  sides  of  (2.21 ),  vie  have 

-’-a]  s-a 

Since  *  f  (£]_ >£2 •  •  •  ■  >£Lj ) 


(2.22) 


Inequality  (2.22)  shows  that  2.  is  a  semimartingale,  where 

0 


E  Z.l1  <E  Z.  <  ...  <  E  Z,  < 

-j+i  -  -a  ■  -1 


(2.23) 


so  that,  according  to  the  theory  of  semimartingales  ^  the  sequence 
converges  with  probability  one,  and  hence  by  virture  of  Eq.  (2.19)  and 
(2.13c)  the  sequence  (c .  -  c'' )  also  converges  with  probability  one  to 
some  random  number  It  remains  to  show  that  P  (£  a  0)  “  1,  It  is 

seen  that  frem  (2.23),  (2.19 )  and  (2.13c)  the  sequence  E(c  .  -c  ’’) 

l) 

is  bounded.  Now  taking  the  mathematical  expectation  on  both  sides  of  the 
inequality  (2,l8), 


MI|C.+1  -  c*n2(  <E] 


d  (=*V  si) 


and  adding  the  first  j  inequalities  together,  we  have  by  deduction 

E{ll-3+i  -  s* II2]  JEflK  -  “’ll2] "  E  4 

-  £  %  e  ta  -/>Tv«) 


(2.2U) 


J 


Since  E 1 1 1  c  ^  -  c^jj^j  i-s  funded  and  condition  (2.13c)  is  fulfilled, 
from  (2.2Uv  it  follows  that 


11— 1:1 


(2.25) 


Using  condition  (2. 13b),  i.e.,  X  if-  °  “  and  noting  Eq.  (2.lb) 


e  <  c 


“  -  ”  i  I  <  F 


E  j  (c  -  c  £  VqI  >  0 


We  deduce  from  (2.25)  that 


E  j  (c  -  £  T  \7Q  j->0  with  probability  one  for  some  sequence  N. 

(2.26) 


Now  taking  E  j  j  |  c ^  -  c"  ||  j-}-  r  with  probability  1,  and  comparing 
(2,26)  with  '  (2.lU)  we  obtain 

-  =  0  with  probability  1 


(2.27) 


Therefore,  algorithm  (2.9)  converges  with  probability  one 


P  Urn  (c  .  -  c  '  )  =  0  }  **  1 

-J  ~  S 


as  well  as  in  mean  square  sense,  i,.e,. 


e)  c  .  -  c  ' 
J*“  |  -3  ” 


(2.28 


(2.29) 


5,  Geometrical  Significances  of  the  Conditions  for  Convergence 


In  the  last  section  we  mentioned  several  restrictions  imposed  on  the 

properties  of  the  sequence  j" as  we-^  as  on  behavior 

the  function  VcQ(x]c).  These  conditions  not  only  guarantee  the  convergence 
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of  the  algorithms  but  also  possess  certain  geometrical  meanings  „ 


n-22 


A.  ^  >  0.  This  is  to  assure  that  the  corrections,  on  the  average, 

are  to  be  made  in  the  right  directions. 

0  as  j-b®.  This  is  to  assure  that  c.  calculated  from 
3  3 

algorithm  (2.9)  will  converge  on  seme  specific  value.  Suppose  we  let  the 
wsesuved  error  gradient,  be  VcQ(xic)  and  the  real  gradient  be  e|  \7cQ(xl£)j„ 
Normally  there  is  random  noise  in  measurement 


VcQ(xic)  =  E  |VcQ(xic)j  + 


3=1,  2,  ... 

Thus  y  Q(xlc.)  1  0  even  if  c.  =■  c*r  For  c.  to  converge  on  any  value 
c  — 3  t  ~3  —  “3 

at  all,  the  condition  >0  as  3'>'=°  must  be  satisfied. 

It  is  seen  that  the  method  of  stochastic  approximation  is  extremely 
noise  resistant.  Random  independent  additive  noise  ^  is  eliminated 
and  does  not  affect  the  final  results. 

OO  _  i  =oo  ^ 

_ _  2  v__  2 

C.  2_  q  .  <  <*>  or  £.  ^.-^0  a3  J-v*.  This  condition  is  to 
j=l  ^  3=J  ^ 

account  for  the  accumulative  effect  of  ^  . .  One  application  of  this  con¬ 
dition  has  been  seen  in  the  last  section.  When  random  noise  ^ .  is 
added  at  each  iteration  step,  algorithm  (2rj)  becomes 


2*!-  V^Vc«<*l2>  **-;)*; 


(2.30) 


Summing  the  above  equation  from  j  =  J  upward  gives 


c  -  c  . 
-®  “3 


"Lr5VQ<il-)  ‘  fi 


(2.31) 


i)-®: 


(2.31)  expresses  the  total  variation  in  c  from  the  Jth  step  onward. 


Since 


j=J 


3  J 


2  "  4. 

w  0  assures  that  the  total  random  variation  27  .  ) 

j=j  2  \  j=j  J-  2 

approaches  zero  as  J  becomes  very  large. 

no 

d.  T3V-.  The  above  conditions  assure  that  c.  converges 

j  =  l  2  0,5  J 

on  seme  value  c  .  s_  0  rr*- °°  assures  that  c  =•  c  Since  this  condition 

—on  — ~  “1  —on  — 

®  j=l  J 

also  implies  2_  (f  if  c  .  approaches  any  value  other  than  c  ,  the 

j=J  2  <*>  2 

total  correction  effect  27"£T-V  Q(£|£.)  is  infinite.  On  the  other  hand, 

.1=2  2  c  ~ 

we  have  no  fear  of  overshoot  because  each  step  is  very  small  as  {f  .—^-0 

3 


when 


j-)1,  Conditions  A~D  state  that  the  rate  with  which  y\  de- 


creases  must  be  ouch  that,  on  the  one  hand,  the  variance  of  performance 
index  vanishes,  and  on  the  other  hand,  the  variation  in  .  over  the 
variation  period  is  large  enough  for  the  law  of  large  numbers  to  hold. 
llif  1  -  /  (£  -  c*)T  \7cQ(xlc) 


E. 


e  < 


-  'I  <  7 


>  0  for  e  >  0„ 


This  condition  determines  the  behavior  of  the  surface  E  I  \7  Q(x|c)l  ■=  0 

x  |  c  -  -  j 

close  to  the  root  and,  consequently,  the  sign  of  the  increments  of  c.„ 

3 

Actually,  if  the  error  criterion  does  have  a  unique  minimum,  the  above  con¬ 
dition  is  generally  satisfied. 

F.  F  ( \7  1  Q(x|c)V  Q(xlc)\  <  d ( c  +  c^o)  for  d  >  0„ 

I  C  c  ~  "  J 

This  condition  requires  that  the  mathematical  expectation  of  the  quadratic 


forms 


Ex  |  VCT  Q(x|c)Vc  Q(x|c)j 


increase,  as  c  increases,  no  faster  than  a  quadratic  paraboloid. 
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III.  TAPPED  DE:  if  LINE  FILTERS 


1.  Optimum  Tapped  Delay  Line  Filters 

Linear  filters  can  be  continuous  or  discrete.  The  optimum  Linear  filter 
developed  by  Wiener  has  the  form  of  Eq.  (1.5) 

*  „,!<"> 


H  (,,) .  JSlL- 
0  ♦jocW 


(3.1) 


Ho(u)  may  be  either  physical  realizable  or  not.  if  it  i3  physically 
realizable,  then  standard  techniques  in  network  theories  can  be  applied 

oh 

to  obtain  Hq(  w)  cons! stint:  of  RI.C  elements  with  or  without  transformers. u 
Another  method  of  synthesizing  a  continuous  linear  filter  of  arbitrary  trana- 
ferfunction  (and,  hence,  impulse  response  )  is  to  represent  it  as  an  infinite 
linear  combination  of  filters 


H(  u)  -  r  C  F  (  •’)  (3.2  1 

i-1 


where  the  functions  (  u)  are  independent  and  together  form  a  complete 
set.  While  an  infinite  sum  is  necessary  to  reproduce  exactly  the  optimum 
filter  response  h(t),  in  practice  it  might  Ko  mere  useful  to  find  the 
best  filter  which  can  bo  constructed  from  a  finite  number  K  of  such 
independent  component.';. 

One  particular  type  of  .  (3.2)  but  discrete  in  nature  is  the  tapped 

delay  line  filter.  This  filter  consists  of  a  tapped  delay  line,  or 
equivalent,  with  adjustable  weights  at  each  tan.  In  thus  case 

!i 

K(a:D  o  c  ,  n;  (3,3) 

k=0  K 


and  the  impulse  response  is 

h(t)  °  y  ( t-T(  ' 

k°0 


(3rh) 


where  T,  «  kT,  T  is  the  delay  increme r  i  \p.v 

k  3 


•er.  delay  line  taps,  C  is 


the  weight  at  the  kth  tap  on  the  filter,  and  6  is  the  Dirac  delta  function. 


The  configuration  of  such  a  filter  is  shown  in  Fig.  2.  denotes  a  delay 


of 


in  time. 


s(t) 


signal 

I 

noise 


xU) 


C 

0 

D  1—0 


I  0  l  > 

"  F1  Ci  \ 

-  o  I— a — 


■I  D 

I  N 


Fig.  2  Tapped  Delay  Line  Filter 


The  signal  obtained  at  a  point  after  the  delay  element  D^  is  of  course 


ft 


n±  ( t)  = 


x(t)  h^(t-r)di 


x(r)  6  (t-  -T  )dr  =  x(t-T  ). 
j  1  i 


(3.5) 


The  weightr  C^*(i  =  1,2,...,N)  which  optimize  any  performance  criterion  may 

be  found  by  using  standard  techniques  such  as  calculus  of  variation  or  by  setting 

the  partial  de: ivatives  of  the  performance  criterion  with  respect  to  the  adjust- 

2 

able  gain  to  zero.  Mean  squared  error  criterion  E{(d(t)  -  z(t)  ]  }  is  used  here 

because  it  is  simple  to  use  as  any  and  most  configurations  are  not  very  sensitive 
,  .  25 

to  error  criterion 

A.  Frequency  ocmain  optimization  using  calculus  of  variations. 

We  art  interested  in  ueterruining  H(v)  minimizing 

F  =  E  { ( d ( t )  -  z(t)]^}  =  d~  ( t)  +  z^(t)  -  2R  (o)  (3.6) 

a  z 

Each  term  of  (3.6)  car.  be  related  to  11  (o)  by  means  of  frequency  integral. 


Since  the  filter  output  is 


Z(t)  =  k=0  Cknk(t) 
and  its  spectral  density  function  is 

*z(u)  "  1*0  k»Q  *2,2  (a))  =  k”0  Vw)  V(uj)  *x(aj) 

i.  K  ■ 


N  N 

E  2  C  C  e 

fc=“0  k=0 


— joj(£— k)T 


the  second  term  in  (3.6)  z  (t) ,  is  given  by 


z2(t) 


2  7T 


N  N 

<f>  (w)dw  =  Z  I  C  C.  r— 

z  .  .  ,  .  I  k  2v 

1=0  k=0 


(jj^Crn)  e 


->(fc-k)T 


dw 


The  third  term  is 


Rdz(0) 


E  d(t)  Z(t) 


1 

2~r 


:  .  ( w )  dw 
dz  ' 


N 

l 

k.°0 


JL 

2n~ 


1>  ,  i  (w)dt.: 

a'  k 


and  final].v 


,2.  .  =  -1- 
d  ^ '  2  n 


•;  ,  (  jj)  d  . 


Combining  (3.9),  (3.10)  and  (3.11),  we  obtain 

r 


~2 —  H  N 

F  =  e  (t)  =  c^ci  ,  2j 


1  :  :o«J"(t"k)T  ] 

dw 


N  1 
■2  Z  C.l-r- 
1=0  k  2" 


.(^)ejjiTdw]  + 


,  'J'd(w)d^ 


0-12  7 


(3.7) 


(3.8) 


(3.9) 


(3.10) 


(3.11) 


(3.12) 


The  frequency  integrals  are  simply  correlation  functions;  that  is, 


and 


(w)  ej^~k^Tdw=  Rx  C^T-kT) 

(3.13) 

GO 

ij  4>d(“)  eJ“tT<i--  B^tfT) 

(3.1U) 

Mow  let 


c>-  =  c:  +  eb  .  ,  k  -  0,  1,  N 


"k  k  ^  k’ 

where  c°  is  the  optimum  and  ^  is  an  arbitrary  constant.  For  c° 
optimum,  F(e)  must  now  have  a  minimum  at  e  n  0.  Or 

N  N 

r  z  Rx^~k)C4(cj? +  e  +  4  (V'e'\) 


dF 

de 


e=  0  k^O  ^  aO 
N 

Ed*  tW><5, 

¥>.  N 


£  =0  k=0 

for  any  and  all  ^  .  Therefore 


*1(1  c  Rx  $~k)  - 


X  C  Rx(£.k)  =  Rdx  (flT)  for  J^-O,  1,  N.  (3.15) 
k=0  ^ 


In  matrix  form 


or 


where 


*1 


R. 


c 


Tn  -1 


c*  =  (R  )  R 


-dn 


=  F 

'rH»  J.V  i ' 

1 

(3.16) 

(3.17) 
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and 


P-1  ■ E  f  Cp  1 


B.  Direct  Differentiation 


Since 


e(t)  =  d(t )  -  Y.  i(t)  =  d(t)  -  ft 

i=0  I 


T. 


p  p  T  -  T  T 

e  (t)  =  d  (t)  ~  2d  )j  c  -  c  1  fj  ^  c 


Taking  the  mathematical  expectation,  we  have 


e  (t)  -  d‘  (t)  -  2d  n  c  +  'c  R, 


Let 


Vc«  • 


.  o 

•a 


P 


omce 


V 


(  T  _ 

I  C  Rn 

„•] 

!  =  R 

c 

+  R., 

1  1 

1 

1 

1  1 

1 

together  with  R  ,  ■=  R 

i  l 


1 


The  gradient  of  e  is 


(3.18) 


(3.19) 


V,  e2  =  -2  d  n  +2  R  T 


1-  i 


(3o20) 


For 


2 


e 


to  be  minimum,  we  set 


0. 
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same  as  obtained  above. 


Then  c*  =  (R  T)“1  R, 
-  — n  -dn 


C.  Derivation  from  Wiener  Filter. 

tt 


Another  derivation  of  c  m  be  obtained  directly  frcm  the  Wiener 
filter.  From  (1.5),  the  optimum  linear  filter  for  additive  noise  is 


(3.21) 


Setting  c  =  c'  in  Eq.  (3.3)  and  combining  with  Eq.  (3.2)  give 

T 


N 


_juo 
s_  j  U)t 


k-0 


( “3 

f  „■) ,  2*12 
l  c  J 


or 


'  jot  N 

eI:Nt 

e 

_3  ',nt 

k  e  / 

(3.22) 


Multiplying  both  sides  of  (3.22)  by  j^e  =  ("e^°...>  ^  _) 

and  integrating  frcm  -*  to  +<*>,  we  obtain 


_1 

2a 


4>*(u)  «  j  (.*)  -&l  (••"*% 

rr  /  _ro 


(3.23) 


Comparing  (3.23)  with  (3*15),  we  see  that  they  are  identical. 


2„  Minimum  Mean  Square  Error  and  Effect  of  !!on-optimum  Settings. 

A.  Expressions  of  mean  square  error. 

The  minimum  mean  square  error  of  the  tapped  delay  line  filter  is 


D-.ii) 


obtained  by  substituting  the  expression  of  the  optimum  filter  into  e  . 
Using 

_  I  >  L  K 

1 


N 


(3.16) 


we  have 


2  ~2,.  ,  „  ,  T  i  •)(  T  D  * 

e  .  =  d  t  -  2  d  r,  c  +  c  R .  c 

min  v  I  —  -  - »?  x 


d2(t)  -  2  d  r,  T  R  T  "1  d 


■1 


,T  -1  T  -1 

+  d  /(  ^  -Rrj  d  rj 

0  T  T  n 

=  d  (t)  -  R.  Rn  R, 

v  '  -d  y  ~<J  -d  rj 

(3.2lja) 

or 

'?  T  x. 

=  d  (t)  -  R,  c 

•dr i  -  - 

(3.2Ub) 

or 

-  d2(t)  -  Yt) 

(3o2ltc ) 

where  ZQ(t)  is  the  output  of  the  optimum  filter „ 


In  terns  of  e2 .  ,  e2(t)  can  be  expressed  as  follows: 
min’  r 


From  e2(t )  =  d^(t)  -  2d,  fj'*'  c  +  c  T  R c' 


(3.19) 


Using  Eq.  (3.l6)  and  (3.2Ub),  the  mean  square  error  is  then  expressed  as 
~2] 


T — 

=  d2(t)  - 

■  2d  /j 

T  ■  ,T 

c  +  c 

c 

2 

=  e  .  + 

min 

V 

£*  ~2qY 

c  + 

T 

c-  R,| 

2 

=  e  .  + 

min 

1  o 

s 

-J 

-1  9  "2 

*  T  . 

'  "1 

0  + 

=  e  . 
nun 

,  -T 
{  2 

1 

(  9  - 

C.“  ) 

(3.25) 
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B.  Effect  on  Minimum  Mean  Square  Error  due  to  Non-optimum  Settings. 

From  (3.25)  the  difference  in  mean  square  error  due  to  non-optimum 
values  of  (  c )  is 

^  F  =  e2(t)  -  ej\n  =  (  c  T  -  c"  T)  <  c  -  )  (3.26) 

Let  c.  =  c!'  +  X  • 

1  i  °  i 

then 


^  f  -  S  T  a  fj  S 

■  f 

<  (N+l  f'  max  J  Sk  .  I  max  I  ■  ft  ■  I  (3^.27  ) 

all  if  1l  i,  o  J  (l3  I 


Thus,  the  error  due  to  non-optimum  settings  is  bounded  if  the  deviations 
of  the  weights  and  the  input  correlation  functions  are  bounded.  Note 


that  for  delay  line  filters  max 

ij  j 


Mi 


Rx(o)  =  Ro(o)  +  Rn(o)0 


C„  Relationship  between  the  minimum  mean  square  error  and  the  number 
of  time  delay  elements  used  in  the  filter. 

From  (3.2l4a)  it  is  seen  that 


2  j2  ( ,  \  D  'T  „T  -1  D 

e  .  =  d  (t )  -  R  ,  R„  R , 

min  d/j  rj  ~a  rj 


Talcing  the  case  d(t)  =>  s(t)  and  noting  that 


d(u)  ^(t)  =  s(t)  x(t-T±; 

“  s  tt )  C  slt-lt  )  +  n(t-it  )Jj 
~  s(o)  s  (,  0  -  i  ^  )  =  Rs ( 1 ^ ) 3 


1X12 


we  have 

( v°> . 

,.R  (T  ) 
x'  n' 

>s(o)' 

e2.  =  R  (o)  -Tr  (a)  R  (T,  )*R0(T  )1 

min  s'  '  s'  s'l  s'nv1 

R  (T,  )  R(o)  R  (T  A 

R  (T,  ) 

|  X  X  X 

x  n-i 

s  1 

R  (T  ) . 

,  x'  n ' 

.  R  (o) 

R  (T  ) 
t  s v  n  'J 

where  R  (T. )  =  R  (T.  )  +  R  (T.  )  (3„2Ud) 

x'  1 '  s'  i 7  n'  1  '  ' 


The  last  term  in  the  right  hand  side  of(3.2ijd)  is  a  functional  of  N, 

the  number  of  delay  line  elements,  and  the  correlation  functions.  For 

any  given  forms  or  values  of  R  (T.  )  a  plot  of  e*\  versus  N  can  be 

x'  1  v  min 

constructed.  It  is  anticipated  that  the  larger  N  is  the  smaller  emj,n 
will  be, 

3«  Adaptive  Tapped  Delay  Line  Filters 

The  above  discu'  's  presented  a  means  of  determining  the  optimum 

values  of  the  gains  *  ;ided  that  the  statistical  properties  of  both  the 

desired  signal  and  the  noise  are  known,,  Unfortunately,  in  practice,  it 
is  not  always  possible  to  know  all  this  information  very  accurately. 

If  only  the  filter  input  and  output  are  available  and  nothing  else,  no 
systematic  procedu/os  cvi  he  found  to  adjust  the  gains/  However,  if  we 
know  something  about  the  system,  then  we  can  develop  seme  algorithms  to 
make  the  filter  optimum.  It  will  be  shorn  that  if  a  desired  signal  is 
available,  or  correlation  functions  of  the  desired  signal,  or  (not  and) 
correlation  functions  of  the  noise  can  be  estimated  within  acceptable 
accuracy,  the  methods  of  stochasi  he  approximation  can  be  employed  to  make 
the  filter  adaptive  to  changing  operating  conditions.  These  changes  may 
be  due  to  variation  in  the  input  signal  or  the  internal  structure  of  the 
filter.  Adaptation  is  accc  ’.cliche d  by  obs -"vntion  of  the  "eaction  of  the 
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filter  to  an  external  signal  or  to  an  internal  variation  with  subsequent 
goal-directed  variation  of  the  filter  parameters  so  as  to  minimize  some 
quality  criterion. 

The  quality  criterion  may  bt  represented  in  the  form  of  the  mathematical 
expectation  of  seme  strictly  convex  (not  necessarily  quadratic)  function 
of  the  deviation  of  the  output  variation  from  the  desired  function. 

For  simolicity  we  shall  use  the  mean  squared  criterion.  Thus, 

I  (c)  =  E  j" Q(d(t )  -  z(t))j  with  Q(e)  =  e2  (3.28) 

For  the  tapped  delay  line  filter  shown  schematically  in  Fig.  2,  we  know, 

x(t)  =  s  (t )  +  n(t)  (1.1) 

It  is  assumed  here  that  these  functions  are  stationary  random  processes. 
The  desired  function  is  t  io  function  obtained  by  applying  an  arbitrary 
operation  on  s(t).  This  operator  may  be  a  differential  operator,  in¬ 
tegral  operator,  predictor,  etc.  It  can  even  be  a  unity  operator  such 
that  d(t)  =  s(t).  e  shall  first  of  all  consider  the  case  where  d(t) 
is  available.  Those  cases  for  which  signal  or  noise  correlation  functions 

are  known  will  be  treated  in  a  later  section’  They  will  turn  out  to  be 
slight  modification  of  the  first  case.  Nonstationary  or  time  varying  systems 

will  be  considered  subsequently. 

For  the  first  case 

1(c)  =  E  f  Q(d(t )  -  z(t)J  i  (3.29) 

N  N 

since  z(t)  =  £  ckl7k(t)  =  Z  ^  x(t-kT ) 

k=0  '  k=0 


I3-.M 


we  have 


N 

1(c)  -  E  |  Q(rt(t)  -  J  ckY)k( t))| 
1  k=0  ’  / 


(3.30) 


Since  P(x)  is  generally  unknown,  algorithm  (2.9)  will  be  used.  For 
2 

Q(e )  «  e  (t ),  we  see  that 

U  Q(x|c )  =  2e  \7  e 
C  *“  c 


Vce  a  vc  -  Z  ckrjk(fc)) 

k=0 


f  ae2  ' 

N 

therefore 

3co 

=  2(d(t)  -  J_  ck  r?  k  ( t ) ) 

^  2 
oe 

k=0  1 

l  ae.  > 

and  the  desired  algorithm  is 


c  . 
— J+ 


1 


(3.31) 


w.'d'  constant 

This  is  precisely  the  LMS  algorithm^used  by  Widrow  "  derived  from  in¬ 
tuitive  reasoning  rather  than  frcm  rigorous  mathematical  proofs. 

It  would  be  desirable  am  instructive  to  give  some  physical  inter¬ 
pretations  of  the  conditions  under  which  algorithm  (3.31)  converges. 
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Algorithm  (2.9) 


(2.9) 


converges  if  the  following  conditions  are  satisfied: 


(a)  lirnTj-o,  ITj-’’  Z  ?Tj  < 
■r>‘“  0=1  0=1 

*  .T 


E  I  (£-£")  Vc  q^(£)  0 


(b)  inf 

£  <ll£  -  £:’l|7>  e  ^  0 


in  the  neighborhood  of  c  „  c  >  0 


(2.13) 


(2. Ill) 


(c)  ^  <  V  T  Q(xlr)  \7  Q(xlc)j  <  d^^c”  +  cTc). 


(2.15) 


The  choice  of  .  which  satisfies  (a)  is  rather  at  our  own  disposal. 

3 

For  example,  y'.  =  -jpg  with  a,  b  >  0  will  definitely  fulfill  the  re¬ 
quirement  of  (a).  The  lemaining  conditions  depend  on  the  surface  of  the 
error  gradient,  wnicH  in  turn  depends  on  the  choice  of  error  criterion  and 
the  Dhysical  system  under  consideration. 

Condition  (b)  is  satisfied  as  long  as  the  function  1(e)  is  strictly 

_'C 

convex.  Since  Q(e  )  has  a  minimum  at  c  =  c it  is  evident  that 

>0  for  c.  >  c!' 

3c.  ii 

l 


=  C  for  c.  =  c. 

l  i 

“  0  for  c.  -  o.’ 

l  l 


(3.32) 


i  -  0,  1,  2,  .....  N 


Consequently 


(c.  -  c!')  >0  for  all  i 

l  l  3c  ^  - 


d-:k» 


and 


e|  (c  -  c*)T  \7c  Q(x|c)) 


>  0 


Condition  (c )  is  satisfied  if 


(a )  8dQ  exists  and  is  uniformly  bounded. 
3e^ 

(b)  s(t)  and  n(t)  are  uniformly  bounded. 


Using  a  Taylor  expansion  about  c  »  c  }  we  have 

N  , 

o  +  y  (c.  -  c*)  |--— 

4.  x  i  3c . 3c  . 
i=0  1  J 


3  Q(c) 

3o  . 

] 


(3,33) 


c=c 


for  arbitrary  j,  v;ith  j  =  0,  1,  2,  N. 

For  the  tapped  delay  line  filter  with  mean  square  error  criterion 

N 

Q(e)  -  Q  (d(t )  -  J  cx >7  i.(t )  ) 

i=0  I 


Therefore, 


3Q(e)  _  5Q 

3c,  "  ae  (-)?.( t )) 


9  Q(e  )  .  3“Q 


3c. 3c  . 

l  j  ae 


f  rJ±(t)rl i(t) 


3.3)4) 


(3.33: 


By  definition 


1 


i(t)  =  s(t-Ti )  =  s(t~i\)  *  n ( t-I i 


(3.36) 


It  is  evident  r.h^t 

are  satisfied. 


3c . 3c  . 
i  1 


is  bounded  if  conditions  (a)  and  (b) 
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Therefore, 


Vc  Q(x|c)  -  ki  I  (ci-ci) 

i=0 


(3.37) 


where  k,  -  k  5”P  .  I 

1  all  i  I  3c. 9c .  i 

i  3 


gives 


Taking  the  inner  product  and  mathematical  expectation  on  each  side  of  (3.37) 
T 


£  (  Vc  Q(x|c  )  Vc  Q(x| c  ) 


N 


i  k-  I  (e..c* )  <  k*  J  (o.  .  4  ) 
i=0  i=0 


(3,38) 


,  /  *T  •>;•  T  > 
=  d(c  c  +  c  e) 


In  practice  conditions  (a)  and  (b)  aiv  easily  satisfied.  Thus  the 
methods  of  stochastic  approximation  can  be  employed  in  a  variety  of 
adaptive  processes. 


1 
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IV.  Adaptive  Schemes  and  Rate  of  Convergence 


Adaptive  Schemes 

A.  An  algorithm  has  been  presented  to  adjust  the  gains  in  the  tapped  delay 
line  filter.  If  the  desired  signal  is  available  to  generate  the  error  gradient, 
the  adaptive  scheme  is  given  by 


l  j+l  -  -fj  +  2  T  ] 

4  ) 

(  4.1  ) 

N 

with  e  ( t  )  =  d  ( t  )  - 

Alt"1 

(  4.2  ) 

k-  ”0 

The  scheme  is  shown  below 

cUt) 


The  complete  adaptive  system  is  shown  in  Fig.  3 


fig.  Adaptive  .. ’m*‘m  with  d(t  )  available 


P 


B.  When  d  ( t  )  is  not  available  but  the  statistical  properties  of  the  noise 
are  known*1, /Algorithm  is  modified  as  follows. 

Using  s(t)=x(t)-n(t)  in  the  expression  of  error  criterion  we 


have 

I  (c  )  =  E  j  Q(e  )j  =  E  |  (s(t  )  -  z(t  ))  j 
=  e|  fx(t)-n(t)-z(t)j  2j 
=  E  j(x(t)  -  z(t))  2  j  +  E  |  n2(t  ) 

-  2  E  j  n(t)[x(t)-z(t))' 


Since 


x(t)  =  s(t)+  n { t  ) 
M 


N 


and 

Eq.  (  4.3  )  becomes 


Z  ck  1 

7k(t  > 

=  2  CkX(t-  Tk  ) 

X 

II 

o 

1 

k-  u 

N 

Z  \( 

'  s(  t  - 

T  )  +  r.(t-T  )) 

O 

li 

|  s  (  t  )  n  i 

(t  )  \  = 

0 

( 

I  (c  )  =  E  /  f  x(  t  )  -  z(t  ))  2\  +  E  f  n‘(t  )\ 


M 


2  E  j  n  ( t  )  j  +  2E  )  n(t)  J  c.  n  (  t  -  T.  ) 


k-  0 


=  EMx(t)-z(t)^  (  -  E  |  n  ( t  )  ) 
'  /  M  >  s  } 


+  2  E 


J  c,  n(  t  )  n(  t-T.  )  ) 


k=  0  *  k  '  j 

(  ■  2~)  im 

=  E  \  fx(t  )  -  z(t  ))  -  Rn  (  o  }  +  2  Z  c  R  (  T,  ) 

'  '  )  k=  0  n  k 


(  4.3 


(  4.4 


where  )  is  the  noise  correlation  function. 

and  e  ^ual  to  s (t ) 

In  comparing  the  algorithm  used  for  the  case  when  d(  t  )  is  available  ,/if 


s(  t  )  is  replaced  by  x(  t  )  ,  we  would  adjust  the  gains  c_  to  minimize  the 
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first  term  on  the  right  hand  side  of  (it.b),  i.e.,  to  solve  the  equation 

1  '  >,  n  ' 

grad  E  )  x(t)  -  z(t)  =0 

f  \  S  ' 

Now  consider  the  rest  of  the  right-hand  side  of  (It.h).  The  second  term 

is  independent  of  C,  and  the  entries  R  (T  )  appearing  in  the  third  term 

are  known.  We  wish  to  minimize  the  sum  of  the  three  terms,  i.e.,  find  the 

solution  N 

grad  E  jx(t)  -  z(t)'(  '  +  grad  2  2  CyVTk  ^  =  0 

k=0 

At  this  point  we  shall  use  a  modified  algorithm  whose  convergence  properties 
and  proofs  are  found  at  Appendix  A.  It  is  shown  that  if  we  let  Q  ■= 
the  algorithm 

Vi  -c}-  vfVW’  (W5) 


also  converges  in  the  same  sense  and  under  the  same  physical  conditions  as 
algorithm  (2.9)  for  t1  j  tapped  delay  line  filters. 

In  (It. 3)  we  can  set 
Q1  =  [  x(t)  -  z(t)] 

Q0  =  ,u2(t)  -  2n(t )  (x(t)  -  z(t)] 

N 


But 


and 


Q„  «  -  R  (0)  t  2  z  C,  R  (T.  ) 
2  n  “  -  k  n  k 

k=0 


0_  =  2R  =  2[  R  (o)  R  (T)  ...  R  (NT)] 
2  -n  n  n  n 


(It. 6) 


Thus,  algorithm  (U.l)  is  modified  to 


C  .  _  ~  C  .  -  y  .  [V  x(t  ’  -  z(t)  2  +  2R  ]T 
-I'l  ~J  3  c  -n 1 

=  C  .  +  2y  n.(x  -  2  .)  -  2y_R 
"J  J  .1  J  J-n 


(h.7) 


The  adaptive  schema  is  shewn  telcw  and  1.1,0  whole  system  is  drawn  in  Fig.  It. 

i>  \ 


z(t  )  - 

— *©- 
\ 

XI  t  ) 


-V 


-K? - ( 


■i  (T.  I 
1 


n-n 


Following  the  Same  procedure  as  before,  we  have  in  this  case  an  algorithm 


— j+ 1  + 2  r,  -Rs  -  *r,  tj, z, 

the  scheme  for  Eq.  (  4.11  )  is  then 


Ml 


*£>— - 


tr 

o- 


i 


while  the  whole  system  is  shown  in  Fig.  5. 


Fig.  5.  Adaptive  system  with  known  signal  statistics. 

In  the  above  schemes  no  distinction  between  continuous  and  discrete 
processes  has  been  made  because  their  connection  is  oovious  .  In  t'.  <■' 

Y-  1 

discrete  case  we  can  set  5  .  =  ~r  'while  its  continuous  counterpart  is 

V"  1  26 

q  (  t  )  =  .  Theorems  concerning  the  choice  of  X  (  t  )  already  exist" 

and  are  not  discussed  here, 

n-  ■ 


2 ,  Rate  of  convergence 


Having  found  an  algorithm  which  converges,  we  shall  Investigate  how  fast 

it  converges  ,  In  other  words,  we  would  like  to  know  the  mean  square  error  at 

each  stage  during  the  adaptation  period. 

From  (4.1) 

c.  =  c.  +  2  X  >?  ,  e 
J  +  l  J  vj4j  j 

and  using  the  expression 


we  get  the  corresponding  matrix  form 


'.'aking  the  mathematical  expectation  of  Eq.  (  4.12  )  and  diagonizing  the 
T' 


matrix  El 


Mi  1 


such  that 


J 


;(l  hf  -i 


-1 

P  A  P 


where  JP  is  an  orthon^rmal  matrix,  and  /\  = 

matrix,  we  obtain 


A°  o 
O  ''A 


is  the  eigenvalue 


W 


S-|  +  1  =  <  1  -  2  r,  Sn  )  £  d  n 

1  1  ■'  I  >  >  j  I  3 

-1  ~ 


=  ( 1  -  2  jr  p  A  -p )  s  ,  +  2  r,  'dj  II  , 


( 4.13  ) 


nV!  .■  31  hg  rt  k:  » 


In  the  abov^f  we  assumed  that  c  is  statistically  independent  of 
Although  c_  can  not  affect  In  any  manner,  the  increment  of  c_  at  each 


D-ll 


stage  is,  however,  related  to  n  by  (Li.l),  Since  the  increment  is  generally 
very  small  and  the  total  effect  involves  addition  of  a  large  number  *T  n  11 
increments,  we  can  assume  oj)  =>  c  fj.  in  a  manner  similar  to  that  used  in  the 
analysis  of  phase -locked  loops* 

Let  us  define 

W  -  p  C,  n.'  =  £  n  (h.llt) 

then  (U.13 )  becomes 

-j+1  =  (1  "  2yj  A)  +  2yj  U.1*) 

Since  d_n '  =■  C*  as  seen  from  (3  ±6),  we  have 

W  -  W*  -  (1  -  2Yj  A)  (W  -  W*)  (14.16) 

How  consider  ar.y  particular  con.oonent  w  of  V  and  for  clari  ty  no 
subscript  or  supuscript,  indicating  the  component  is  used.  Then 

W  -  W*  =  (1  -  2y  A)(W,-  W*)  (U.1Y) 

J  J 

Using  Eq,(L.lV  ,  recursively  gives 

w.  =  (\-\  -  ’/)  '"l  (i  -  2y  .  x )  +  w*  (14.18) 

J  k=i  J 


Viterbi,  .A.J.,  Principles  of  Conor-  no  Ccrxunica*. J. on ,  raw  dill  rook  Co. 
Hew  Ycrk,  1966. 
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We  shall  now  find  w^ 

T 

From  (4.1)  and  taking  the  product  of  c  and  £  ,  we  obtain 

Vi  '  %  +  2h  h  V  +  2h  ej  %T) 


T  .  T  ,  T.  ,  .  2  2  T 

-j  -)  +  Tj  h  (%  Ai  +  %  h  }  +  4h  h  At 


(4.19) 


Since 

T  T 

e,(c  h.  +  n.  c.  ) 
j  -j  -j  ~i 

-  Mj  -  i)T  Cj)  (£j  +  a,  CjT) 


%  dAT  +  dJ  ^3  £jT 


T  t  T  T 

-  cj  c  n,  n.  -  n,  n.  c  c 

-j  ~j  j  ~j  “j  “J  ~j  j 


Note  A  B  T  +  B  A  T  =  ’{ABT]S 


where  s  denotes  the  symmetrical  part  of  a  matrix.  For  example,  if 

r 


A  = 


all  a12 


a21  a22 


then 


AS  = 


2  (311  +  all)  2  (a12  +  a21)  ! 


2  <'a21  +  a12^  2  ^a22  +  a22' 


we  have 


,  T  .  T, 

p  ( r  n  4*  r,  c  ) 

J  -j  -J  -j  -j 


:  ( 


2  I  R  c*  c 

.  n  j 


t)  s 

l/  -2  {K 


S 


{K  c.M 

n  J  j  ) 


2{b^*  -Sj)  £/ 


(4.20) 


jJ-4  (j 


Taking  mathematical  expectation  on  both  side  of  (4.19  and  using  (4.20)  yield 


c,T,  -  c.  c.T  +  4v.  \R  (c*  -  c.)  c,T 

j  i  — n  —  “i 


^j+i  -j+l  ^ 


rl'  ^3 


2  2  T 

+  4tj  ej  hh 


(4.21) 


For  large  J  ,  the  following  approximation  can  be  made 


2  T  _  2  T  2 

6j  ^j  6  “in  -^j  e  min  — n 


(4.22) 


(4.22)  can  be  viewed  as  a  Taylor  series  expansion  around  the  optimum  point 
and  with  higher  order  terms  neglected  for  large  j 

Therefore,  (4.21)  becomes 


%1  fj+1  °  %  +  *Yj  \  ^  (^*  -  £>  ±  jj 


2  2 

+  4y,  e  .  R 
j  min  — n 


(4.23) 


Using  the  transformation  c  =  P  ^  W  as  defined  In  (4.14),  we  can  change  (4.23) 
to  the  form 


T~  T  -1  -1  T  )  -1 

W. , ,  W,  ,  =  W.  W.  +  4y.P  P  '  P  P  (W*-  W)W  P  '  P 
-j+l  -j+l  -j  j  j-  -  —  -  - j  - 

2  2  -1  .  -1 
+  4y  P  e  .  P  •  P  P 
'j  -  min  -  - - 


And 


W  WT  <D 
-j+l  -j+l  \ 


T  ■  °  '  4y.  i  '■  (W*  -  W)  WT  \  D 


IW.  W.  '  +,,_, 

( — j  — J  j 


.  ,  2  2  , 
+  4Yj  emln 


(4.24) 


In  the  above  D  denotes  the  diagonal  elements  of  a  matrix.  These  elements  have 
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the  desired  form  w2  >  which  can  be  exPressed  as 


Vl  =  Wj  +  4YJ  K  (W*  '  Wj}  Wj  +  4yJ  X  "min 


(A. 25) 


(1  -  4y,X)  w  2  +  4y  X  w*  w,  +  4y  2X  e2 

j  j  j  j  j  min 


From  (4.16)  we  have 


Wj+1  W  j+1 


(1  -  4y^  /\  )[^  ]  +  4y^  w*  w 


—  T 

h 


(4.26) 


Let  0J  -  (^  -  Wj)  (Wj  -  Wj)T  =  (W  WT)j  -  Wj  WjT 


Substrating  the  diagonal  terms  of  (4.24)  from  those  of  (4.26),  we  obtain 
Vl  =  U  “  S  3  -9j  +  4Yj2  *  6min 


(4.27) 


D  2_2 

Since  0,  has  the  elements  Wj  ”  wj  ’  we  sce  tbat  £°r  any  particular 


■J 


comDonent  of  0 


D 

j  ’ 


Vl  '  Vl  =  °j+r(1  -  /,YjX)  °j  +  emin 


(4.28) 


Iterating  backward, 


Vi =  9i  k^i  (1  ■  4\A) 


+  4Aemin  Jl  \  J  k+1  (1  "  VA) 


But  0  =0  because  w1  =  w  , 

1  L  1 


2 


J  2 

0 .  , .  =  4X  e  ,  I  y,  tt  (1  -  Ay.X) 
J+1  ">1"  k=1  ,=R+1 


(4.29) 


(w  “  w  2  2  J  2 

j+1  j+1)  =  4ye  1  y  it  (1  -  4y  X) 

k=l  k  £=k+l  A 


(4.30) 
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Several  sp<  cial  cases  will  be  considered. 

1 


(1)  Setting  y  =• 


j  2 (j+1) A 


(4.31) 


Th.s  is  a  legitimate  expression  as  Yj  defined  by  (4.3i)  satisfies  all  the 
required  conditions  for  convergence. 


Note  j  j 

’  (1  '  1ST  ’  * 

k°l  k=l 


k+1  j+1 


(4.32) 


(4.18)  gives  us 


'j+1  j+1  V"1 


(w.  -  w*)  +  w* 


(4.33) 


Note  also  that  (see  Eq.(B.8)  of  Appendix  B], 


j 


J 


tt  (1  -  4y  A)  =  it  (1  - 


i=k+l 


£=k+l 


2  .  „  J\+1T 

4+1  '  ,,.,,2 

(j-7-O.) 


(4.34) 


j 


j 


tt  (1  -  4y  A)  =  Z 


k=l  i^k+l 


(k+1) ' 


k=l  4A2(k+l)2  (j+1)2 


j 


J. 


4A 


k=l  (j+1)  4A  (j+1)' 


(4.35) 


T-hus  (4.30)  gives  us 


(wj+l  '  Wj+1)' 


'min 
A  ’ 


(1+1) ‘ 


(4.36) 


As  derived  in  (3.25),  the  mean  squared  error  at  any  time  is  given  by 


e2  =  e2  +  (c  -  c*)T  R  (c.  -  c*) 
j  min  '-j  •  n  -J 


=  e2,  +  (c,  -  c*)T  P-1  ^  P  (c,  -  c*) 

min  ~j  “  J 


=  e  .  +  (W,  -  W*)  \  (W,  -  W*) 

min  “j  -  "j 


(4.37) 


The  expected  difference  between  the  mean  squared  error  at  each  stage  durinR 


the  adaptation  period  and  the  minimum  mean  squared  error  is  then 

D— 1‘) 


r»  ) 

E  e  t  , .  -  e 


L 


'j+1  min 


E  '  (V  -  W*)  A  (W,^  -  W*){ 


j+1 


J+1 


N 

E  .  Z  X  (w  -  w*  ) 

(  1=0  1  j+1>1  1 


Z  x,  (w 


1=0 


i  '"j+l,i 


-  rj* 


,  2 


J 


Cut 


(wj+1  -  W*)2  =  (wj+1  -  uj+1)2  -  wj+i2  -  2  w*  wj+1  +  W*2 


Using  (4.33)  and  (4.36),  we  have 
-  '  2 


(w  i  -  w*)‘ 


~min  j  -t-  1  (w  -  w*)‘ 

\  7  ?  1 

(j+1)"  (j+1)" 


(4. 381  becomes 


E  >  e2 


i  j+1 

v 


2  v 
’  "min  \ 


— j -  r  e2  +  — — ■  r  X  (w  -  w*)‘ 

<'j+i)2  k=0  mJn  (j+1)2  k=0  k  1,k  k 


— ^ — ,  (U  +  1) 

(J+D‘‘ 


+  — — „  (c,  -  c*iT  H  (c.  -  c*) 


'mln  '  (j+1)2  '■'1 


-n  1 


Thu  last  step  is  obtained  from 


N  ?  T  T 

Z  X.  w;  =  W  A  w  =  c  R  c 

k=o  1  1  -  -  "  -  -r'  - 


Thi’.o  for  large  j  , 


E  e 


9  )  (N+l) 


2 

Lniin 


j+1 


'min 


j+1 


f4.40)  is  the  desired  expression  for  the  rate  of  convergence.  For  large 
the  mean  squared  error  decreases  approximately  as  the  first  power  of  time. 


(2)  Setting  Y . 


_ 1 _ 

2 (j+1) 


(4.38) 

(4.39) 


(4.40) 


(4.41) 

j  » 

(4.42) 
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The  choice  of  y  defined  by  (4.31)  requires  some  a  priori  knowledge  about 
the  signal  and  noise  properties.  Otherwise,  if  the  correlation  matrix  is 

not  known,  the  eigenvalues  >,  cannot  be  determined.  The  arbitrary  choice  of 
Xj  defined  by  (4.42)  will  be  studied. 


From  (4.18)  with  y 


j  -(j+1) 


we  have 


w,,,  =■  (w.  -  V*)  TT  (1 


’j+1  '"1 


k=] 


j+1 


)  +  w* 


(4.43) 


Bat 


j 

IT  (1  - 

k=l 


j  +  1 


)  - 


r(i+i-x) _ 

(j+1) ! r(2-A) 


r (2-x) (j+i) ' 


for  j  >>  1  and  j  >>  A 


(4.44)* 


Thus 


(w^  -  w*) 


i+1  T(2-X)(j+1)/ 


+  w* 


(4.45) 


Note  also  that 


j 

TT 

£=k+l 


(1  -  4YiX)  = 


l  n  2X  „  (k+i)2A 

'  2+1  2X 

2=k+l  1  (j+1) 


(4.46) 


Therefore, 


j 

v 

k=l 


tt  (1  -  4y^X)  = 


£=k+l 


jE  _i_  (k+d2X 

k=l  4(k+l)2  (j+l)2X 


4 (j+1) 


2X 


j 

1  (k+1) 

k=l 


2X-2 


Using  the  formula  (No.  29.9,  Tables  of  Integrals  by  Dwignt) 


2  u 
u=l 


p  _  n. 


p+i 


p+i 


p 

2~  II 


n  ‘  1 

+  tt  — —  p  n 


P-1  _  i 


30  4  i 


p(p-l)  (p-2)nP  ^  i 


*  Derivation  appears  in  Appendix  B. 
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we  car.  let  k+1  =  n  ,  p  =  2A  -  2  ,  n  =  j+1  ,  and  obtain 


(k+1) 


2X-2 


k=l 


J+1  j+1 

T.  u  P  =  2 

u=2  u=l 


u  P  -1 


(4. 47) 


-  -1  +  +  yj-  ( 2A— 2 )  (j+l)2^3 

then  becomes 


+ - 


(4.48) 


j  2  .) 

2  Y.  it  (1-Ay  X) 

k=l  H=k+1 


_J _ [-!  +  U-tir':1 

4(j+l)2X  2X-2 


-1 


(j+1) 


] 


2X  (2X-2)  j+1 


]  for  large  j  . 


(4.49) 


Substituting  (4.49)  into  (4.10)  and  combining  with  (4.43)  yield 

2  t 

min  1  (  _ w  L  >  2  ) 

2A  -  2  j+1  /,,,.2X  k  „2 ,  „  .  .  ~  emin  3 


(Wj+1  -  «*)■ 


(4.50) 


(j+1) 


r  (2-a  ) 


and 


2  _  2 
"j+1  emin 


N 

L' 

k=0 


X.  (w.  ,  .  -  w*  ) 2 
k  ]+l,k  k 


N 

k=0 


“min 


(w 


■ — ..  + 


w*  )‘ 
k; 


(j  +  U  (2AR-2;  (]  +  1)2X. 


Uk _ 

f2(2-2Xk) 


(4.51) 


(3)  Yj  ~  Y  =  constant 

The  expressions  for  Yj  defined  by  (4.31)  and  (4.42)  satisfy  the  conditions 
for  ..anvergence  as  stated  in  (2.13).  In  these  cases  the  v  and  thus  the  gain 
Increment  Ac (  become  smaller  and  smaller  ns  time  j  proceeds  during  the  adapta- 

j 

tion  period.  It  is  anticipated  that  the  rate  of  convergence  will  he  Increased  if 

29 

a  small  constant  vilue  is  set  fur  y  .  As  shown  by  Comer  ,  the  algorithm 

with  constant  y  has  comparatively  little  noise  resistance,  furthermore,  in  the 

2 

presence  of  measuring  error  with  variance  o  ,  convergence  in  the  usuai  sense 


does  not  occur,  but 


Urn  E  I 

j-Ko  1 


C,  -  C* 


j<  F  (  (c 


and 


F(^q  ,  o  )  -  o  is  y  o 

Now  we  shall  study  the  rate  of  convergence  when  y  is  a  constant. 
From  Eq.(4.18)  we  see  that  with  y  =  y  =  const  , 


w  =  (w  -  w*)  it  (1  -  2yA)  +  w* 
J  k=l 


=  (1  -  2yl)J  (w^  -  w*)  +  w* 


Since 


a  +  ay  +  ay '  +  ....  +  ay 


n-1  _  a(l  -  y  ) 
1  -  y 


We  can  obtain 


’  (1  -  4y\)~k  [O  -  4y\)"(J'1)-l] 

k=l  V'A 


Thus 


j  2  j 


J 


1  y  ‘  v  '!  -  4y\)  -  y“  (1  -  4y\)  ^  k  ^ 

k=l  £.=k+L  k=l 


2  i-i 

y  (1  -  4y>.)J 


(1  -  4,.U 


k  =  l 


2  1  n  / 

=  y  ^  [1  -  4y\)  ] 


and  (4.30)  becomes 


(wj+l  "  Wj-H)2  =  cmt„  yI1'(1'M)J'1] 


(4. S3) 


(4.54) 
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'he  mean  squared  error  is  Chen 


"J+l 


N 

+  Z 
k=Q 


"'min 


1  . 


N 


Vn  Y  *  xk  IJ--C1-4Ya)J-1] 

k^O 


\  <»i,k  -  ”*k>2  «  -  2'*k>21 


(4.55) 


It  is  seen  from  (4.35)  that  if  the  error  is  to  decrease  at  all,  one  basic  require¬ 
ment  should  be  met,  i.e., 

0  <  1  -  4yX  <  1  with  y  >  0  (4.56) 


which  implies 


0  <  y  < 


_1 

41 


(4.57) 


max 

X  is  the  largest  eigenvalue  of  the  correction  matrix  R  .  Thus 
max  — n 

Y  =  constant  cannot  be  set  at  will  if  st  ility  of  the  adaptive  loop  is  to  be 
maintained. 

T\e  rate  of  convergence  has  been  obtained  so  far  mly  for  the  algorithm 
with  the  availability  of  a  desired  signal  to  generate  the  real  time  error 
function  e(t)  .  New  we  shall  compare  the  algorithm 


£j+l  “  £j  +  2>j  lij 


£.  +  2Y<  b.  -  2y.  z,  .. 


— i 


Mj  j  -j  "  ~YJ  1  ~J 


(4.58) 


where  s,  replaces  d,  for  the  desired  signal  with  the  other  two 

j  J 


£J+1  ‘  £j  +  4  -  2<J  zj 


Vi  ■  h  *  lyi  -1  UJ  "  'J: 


-•I',  n 


J 


(4.59) 

(4.60) 


when  signal  or  noise  correlation  functions  are  used. 


I  Mil 


Taking  mathematical  expectation  on  both  sides  of  (4.58)  givo^s 


'  %  +  2tj  -j  %  -  2b  h  i) 


But 


s(t)  +  n(t) 

sn  =  E  '<  s(t)  ]  s(t-T'  +  r.(t-T)  j  t 
I  s(t-NT)  +  n(t-NT). 


R3(o) 

i  VT> 


R  (NT)  | 
8 


and 


z .  n . 


n.n.c  =  R  c. 
-j-j-j  -n  ~i 


we  thus  have 

£j+l  “  (1-2^  V  %  + 

Taking  the  average  on  both  sides  of  (4.5'y)  gives 


c,-h  ■  cj  +  ^  5,  -  Jvj  ».  a, 
which  is  identical  to  (4.64)  by  virtue  of  (4.62). 
Taking  tbo  average  on  both  sides  of  (4.60)  gives 


-j+1  -j 


c,  +  ZTj  ij  (Kj  -  V  -  2t  ,  E_, 


j 


But 


Oi 


rii  xj  -  8n ' E 


s(t)  +  n(t) 
s(t-T)  +  n(t-T)  j  j  [s(c,  +  n(t)]  _  R 

s ( t -NT ) +  n(t-NT)  , 


R_(o)  +  Rn(o) 

i 

=  j  F.  (T)  +  R  (T)  j 
s  n  t 


R  (NT)  +  R  (NT) 1 
s  n  .  < 


R  (°)  ' 
n  i 

1  I 

R  (T)  j 

-  n 


IR  (NT) 

LI 


R  (o) 
s 

R  (T) 
s 


R.(NT)  j 


(4.61) 


(4.62) 


(4.63) 


(4.64) 


(4.65) 


(4 . 6o) 
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(4.65)  can  then  be  reduced  to  (4.64). 

However,  (4.64)  is  just  (4.13)  if  d(t)  ic  replaced  by  s(t).  We  therefore 
can  conclude  that  for  filtering  prcblem  where  d(t)  ■>  s(t)  ,  the  expected  values 
for  the  gains  at  any  stage  are  given  by  the  same  formula,  i.e., 


j 

W,  .  ,  =  (w.  -  W*)  TT  (1  -  2v  i)  +  tf* 

-1+1  1  k=1  J 


(4.67) 


which  is  valid  for  the  transformed  gain  components. 


Let  us  now  consider  the  variation  of  c  for  the  other  two  cases. 

Taking  the  product  of  each  side  with  its  transpose  in  (4.59)  gives 

£j+l  -j+1  £j  %  +  2Yj  fj  ^  *  2yj  Zj  Hj 

t  2y  R  c  T  +  4y  2  R  R  T  -  4y  2  z  [R  ]  fn],T 
j  ~s  ~j  j-s-s  J  j  s  j 

,  T  .  2  „  T  ,  .  2  2  T 

-  2Yj  ^  -  4Yj  Zj  i,  5,  *  4t,  z,  n,  nd 

■  %  h  +  2b  +  % 

-  ^  Zj  (Cj  4  ♦ 

-  4y  2  z  (R  n.^  +  n.  R  *) 

j  J  ~s  “j  “J  ~s 

2  T  2  2  1 

f  4y,  R  R  +  4-y .  z  n .  n , 

j-s-s  j  j  -j  -j 


T  ,  yr  TvS  !  (  r  TvS 

C.  C,  +  4',  (R  c  )  -  4v  (r  n  c  c, 


"j  -j  ’j  ---j 

9  T  : 
.1  c .  n.  n 

J  -s  -j  -j 


Tj  "J  -J  ~J  ^  ' 

2  1 
U  R  R 
j  ~s  -s 


-  8T.2  (R  C,\  n,V  -  4y  2  ^  KT  .  V,,2  ,  2  r,  »T 

-1  —  c  —  n  — H  — 1  1  — Q  — Q  J  j  — J  — J 


When  average  is  taken  on  both  sides,  we  have 


T  T  /  j  T  I 

c.^,  =  c  c  +  4y  Re.  -  P.  c,  c  \ 

-j+1  -J+1  -j  -j  j  -s  — j  -n  -j  -j  \ 


-8'j’  %T  */  "  ‘Yj 


t  2  2  T 

R  '  x  4y  z  J]_  n. 

~s  j  j  j  J 


(4.68) 
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Similarly,  if  we  take  the  product  of  each  sides  with  its  transpose  in  (4.58)  , 
we  have 


T  T  .  T  T. 

%1  %1  •  CJ  +  2y3  %  9  3  %  +  8J  ^  > 

'  2r3  *3  (%  V  +  %  c3T> 

'  8Y  “3  V  +  ‘V'jSV 


,22  T 

4y,  ^  r  Hi 

J  J  ~J  ~J 


(4.69 


We  shall  see  that  (4.68)  and  the  average  of  (4.69)  are  equivalent  by  virtue  of 
the  following  terms. 


T  ,  T^-dT,dTT 

(1)  c.  s.  n.  +  s.  n.  c.  *  c,  R  +  R  c, 
~j  J  -j  j  -a  -j  “j  -s  s  -j 


(2)  Zj  (83  kJ) 


Zj  'j  ~j  ^  +  "j  "j  9-J 


2Zj  9.1  ^ 


(3)  Rs  Ua  "  SJLj 


6  j  nj  "i 


The  last  expressions  r.re  approximately  correct  if  the  number  of  taps  is  large. 

T 

Thus  c  i-f-i  Cj+1  derived  from  (4.58)  or  derived  from  (4.59)  are  equival¬ 

ent.  Similar  steps  can  be  applied  to  (4.60).  In  conclusion  we  can  state  that 
the  rates  of  convergence  are  the  same  regardless  of  the  choice  of  algorithms. 
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V.  Adaptive  Tapped  Delay  Line  Filters  with 
Time-varying  Parameters 

The  adaptive  schemes  using  the  methods  of  stochastic  approximations  have 
been  studieu  for  tapped  delay  line  filters.  An  implicit  assumption  made  so  far 
is  that  the  system  und^r  study  is  time-invariant  and  all  the  signals  and  noise 
are  generated  from  stationary  sources.  Although  ergodicity  of  the  process  has 
not  been  required,  wid  .-sense  stationarity  is  implied.If  the  system  itself  or  the 
input  signals  are  nonstnticnary  or  time-varying,  the  adjustments  made  for  mini¬ 
mizing  certain  error  criteria  may  not  produce  the  desired  effects.  Suppose  that 
the  rate  of  parameter  variation  is  faster  than  that  of  convergence,  we  can  never 
expect  to  have  the  alco'ithn.:  converge  at  any  time.  However,  if  the  rate  of 
parameter  variation  is  slow,  we  can  estimate  its  effects  in  a  qualitative  fashion. 
Let  us  say  that  Cj)  ( t  )  is  a  slowly  va.y  q  ime  function  if  the  relative  change  in 
its  value  in  any  interval  of  length  t  -  —  is  small;  here  w  is  the  mini- 

u  0 

o 

mum  frequency  o'  the  natural  oscillation  of  the  system.  If  the  transient  behavior 

is  aperio-.de  for  ary  initial  conditions ,  the;  function  Cp  ( t  )  is  said  to  be  slowly 

varying  when  it;;  change  is  small  in  comparison  with  the  relative  change  of  the 

output.  The  term  "slowly  varying"  used  throughout  this  report  is  defined  in  the 

above  sense.  Tl.o  stn.istic.il  properties  of  the  delay  line  filter  will  be  studied, 

Tor  stationary  a.m  nor.  at  ionary  input  signals  the  results  scorn  trivLl  as  a  delay 

element  do  s  no  han  .  any  statistical  properties  at  all,  but  for  iV.o  time-varying 

case  the  r.rthoc  developed  gives  us  some  insights  about  the  system. 
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1.  Statistical  properties  of  delay  line  filters 

The  statistical  properties  studied  here  refer  only  to  the  auto  correlation 
functions  and  variance  of  the  output  as  a  measure  of  the  accuracy  of  the  system. 
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Some  other  properties  like  output  distributions,  probability  density  functions, 
etc.  are  not  considered. 

A.  Stationary  case . 

Let  us  first  of  all  consider  a  single  delay  element.  The  input  and 
output  are  related  by 


and  the  transfer  function  is  given  by 

n  /  i  \  y(|o)  —  J  LJ  T 

H  ( j  u  )  =  1  =  c  e 

x  ( J  a  ) 


(  5.1  ) 


If  x (  t  )  is  a  stationary  random  function  with  its  covariance  function 
function  given  by 

Rx  (  T  )  =  Dx  e-^lTi  (5.2  ) 

The  above  expression  corresponds  to  a  Markov  process  and  its  spectral 


density  function  is 


'  f° 

D« - L 

2  d  Dy 

^C2 


Dx 

WT 


^  _p(jri  -jut 

e  dt 


•fo 

-  1  ujT  ) 


dt 


(-dt 


(  5.3  ) 


The  output  spectral  density  function  is  accordingly 

6  ( «*> )  =  4>  ( u )  I  k(  j  ^> )  1 2  =  ■ 2  f  D*  c2  = 

7  Tx  I  I  u  2  +U) •  2 


2  U  c  2  Dy 
ck  2+“  2 


(  5 .  ‘i  ) 
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The  output  variance  is  tl  en 


D  =  _L 

y  2*  J 


rCD 


4*y  ( 


2  U  c2  f00  dw 

2~7t  x 


y 2 


=  Dv 


C>(  c' 

~tT 


.  2 


A 


=  c2Dx 


2c2 


(  5.5  ) 


as  we  expected . 

A  tapped  delay  line  filter  consist  oi  several  delay  elements  ,  gain  constants, 
and  a  summer  so  that  the  transfer  function  of  the  filter  is 

H(  jcO  )  =  X  ci  f'j  Tl  (  5,6  ) 

i=  0 

Direct  combination  of  (  5.6  ),  (  5.5  ),  and  (5.4  )  yields  the  output 
spectrum 


<p  ( )  =  a 

1  y 


N' 


-i  (T-T  ) 

k-n  ‘t  ‘k  -  '  k 


(  5.7  ) 


and  the  output  variance  ^  ti,  r  , , , r i v ,  transform  of  (5.7) 

-  .Ur  -  kl I 

D  =  D  ‘  *’ 

y  x  ],=n  .  -i  '■ 


(  5  ,b  ) 


B.  Nonstationary  case 

Suppose  that  the  input  is  ,i  nun -stationary  time  function  with  the 
correlation  function 

tyt,  t'  )  =  crx(  ,.( t  )  e'  d  ,t_t!  ( 5.9 ) 

The  random  function  x  nay  re  <  .  pressed  by 

>c(t  )  =  CT  (*)>;.(  t  )  (  5.10  ) 

X  1 

whers  x^(  t  )  is  <3  SKUion  '.i  y  !  ;  tins  ^unction  wit*!',  correlation  function 

pi  von  bv  (r>.2) 


Ni  <T 


(5.11  ) 


I  )-!.(! 


and  spectral  density  function  given  by  (  5.3  ) 
_  2 


(h  .  (  u  )  =  JL£ - 

rxl 


(  5.11  ) 


Assuming  that  the  standard  deviation  (J“  ( t  )  of  x(t  )  can  be  approximated 
closely  enough  by  the  exponential  function 

OV"  >  '  CTo^*  I5-12  ' 

According  to  27  ,  a  function  x(t  )  can  be  put  into  an  integral  expansion 

of  the  type 

N  ftn 

(  5.13  ) 


x  (  t  )  =  (  t  )  +  2T  vf  (  a)  )  xf  ( t,  at  >  dun 

r  =  1  J-oo 

where  m  ( t  )  is  the  mean  value  of  x(t  ), 

X 

v  (w  )  are  uncorrelated  white  noise 

and 

xr(t,1''  )  are  coordinate  functions  defined  by 

x  (t,  w  )  =  1  C  (  ui  )  eU  k  +  1J>  )  1  (  5.14  ) 

r  r  k 

k  =  l 

r  =  1,  2, - ,  N,  C  ^  (  w  )  are  coefficients. 

While  the  output  can  also  be  represented  by  an  integral  canonical  expansion 


like  >:(  t  )  with  the  coordinate  functions 


Y  (  t  <  w  )  =  1 


k  =  l 


Jx\. 


)H(/«.+)Je  k 


(M  i,  +  jco  )  t 


( 5.15  ) 


r  =  1,  2, - ,  N 

The  general  formulae  for  the  convariance  function  and  dispersion  of  the 


output  have  also  been  provided  by  (  p.  239,  Ref.  27  ^ 


N  rco 


Rjt,  )  =  2  i. 

r  =  1  J 


*  . 


to 


Cr(-  )  Z  Cr,-  Cr  <  “> 
k,J?=  1 


H  (xl  K  +  j  1,1  )  H  (/(  +  j  ui)  e 


Wt+¥  +  )"  n-t'  ) 


dw 

(  5.16  ) 
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N  r°°  s 

Dy(‘l=  2  r.  Gr(“'  |Z  °rk<»  ><£<4kt 

r  =  1  /  !k=  1 

~  kO 


Mki 


j  OJ  )  e 


du 

(  5.17 


where  C^.(  u  )  are  white  noise  intensities  . 

In  our  present  case  N  =  1  ,  and  the  coordinate  function  is 

(u  +  _1w)t 

x(  t,  CD  )  =  o  e 
o 

and  the  filter  transfer  function  is  modified  as 

N 


(  5.18  ) 


H  {sU  +  j  u  )  =  r  C,  e‘J  +  u  )  Ti 
i=  0 


Therefore,  using 
function 

Ry(t,  t'  ) 


(  5.19  ) 

(5.16  )  and  (5.17  ),  we  have  the  output  correlation 
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J 
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2  M  (  t  +  t '  )  -o<  1 1  -  t  x  |  N  ? 

CT  e  e  £  Ci 

°  1=0 


(5.20  ) 


and  the  variance 


D,«,  =  !  * q < 1 ' ci2 

Y  i=  0  A 


N 

=  Dx(t)  I 

i  =  0 


d  j' 


-  oo 


c/  ^  +  2 


(  5.21  ) 


2  2  2 /X  t 

where  D(t)=(T  (t)=CT"e’  ,  the  variance  of  x(  t  )  . 

x  :<  o 


C.  Time-varying  case . 

It  is  a  very  important  case  d’ -  n  a  linear  system  is  not  stationary  throughout 
its  total  operating  time  but  its  i  bar  is  cl  e  to  it  for  a  comparatively  short 
period.  II  such  a  system  receive  or  tout  which  is  near  to  an  exponential  function  , 
then  the  output  is  also  near  to  an  exponential  function  at  the  end  of  transient  behavior 


I )-  '  J 


The  technique  used  here  is  confined  to  the  situations  where  only  slow  time  varia¬ 
tion  is  involved. 

Consider  a  single  delay  element 


r 


yit4— i- 


i 

L 


1 

I 

-+ 


^  y (t) 


Since  y  (  t  )  =  C  x  ( t-T  ) 

- 

therefore,  Y(  j  w  )  =  C  X(  J  )  e 

e+  ja)T  Y(  j  u,  )  =  C  x  (  i  UJ  ) 

Or  in  time  domain 

e  +  tT  y(t)  =  cx(t)  (  5.22  ) 


where 


is  an  operator  represented  by 
cO 


?T=  £  -X.'  _iL  „nh  -t,--  A 

dt-  ^  dt 


=  0 


(5.23  ) 


'pT 

At  a  first  glance  e  appears  to  be  a  strange-looln-.g  operator. 

1)  T  co  t 

Actually  e  y(t  )  =  ^  T  1  dJy(t)  is  just  the  Taylor  series  expan- 


i=  0  j  i 


dt  ‘ 


sion  of  y(t  +  T  )  around  T  =  0  .  The  role  of  -p  played  here  Is  clear. 

At  ,A  t 

Let  x(  t  )  =  e  with  X~  M-  +  and  y(  t  )  =  ?.( t,  \  )  e 


then  Eq.  (  5.22  )  becomes 

f  tJ 


j-0  J.' 


(z(t,A  )  e/')  =  ce 


A  t 
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z(  t  ,  A  )  [e  j  e 


*T')  M  .  A 
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A  )  =  c  e 
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.  t  v  .  A  T  A  t  At  00 
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I  Xi  ll_fz(t,A))  =  ceAt 

J  -  1  j  I  -i  ^  > 
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(  5.24  ) 


V.'e  shall  develop  a  procedure  to  approximate  z(  t,  \  )  .  In  the  first 
approximation,  we  neglect  the  derivative  of  the  slowly  varying  function 
z(  t,  \  )  and  obtain 

AT 

z  ( t,  A  )  e  =  c 


-AT 


At  A(t-T) 

and  y1  ( t  )  =  z  ( t,A  )  e  =  c  e 


thus , 

Zj  ( t,A  )  =  c  e 

A  t  A ( t-T  ) 

(  5.25  ) 

as  we  expected  since  y(  t  )  =  c  x(  t-T  ). 

In  the  second  approximation,  the  first  derivative  of  the  slowly  varying 
function  z(t,  A  )  is  taken  to  be  equal  to  the  derivative  of  Zj(t,A,  )  from 
the  first  approximate 

dZlU'M  _  c  (  5.26  ) 


c)  z  (  t ,  A  )  .. 
3  t  - 

where  c  - 


t 

a  c(  t  ) 
dt 


AT 


We  then  get  the 


z,  (  t ,  A  ) 


wing  equation  for  the  second  approximation 


i-  T 


a  zi  ( t,A  ) 

_ t _  =  C 

2)  t 


AT  .  -at 

z0(  t,  a  )  e  c  -  T  c  e 

z2(l,A)=  —AT  ^  c"  T  c  e”  ^  )  (5.27) 


the  nth  approximation  of  .  (  t ,  A  )  it 

zn(t,A  )  =  (c  -  Z 


d  c(t  ) 


i=  1  i 


cl  t 


i 


depending  upon  t.  i.  .or  n  to  which  the  time  derivatives  of  c(t  ) 


c  ist. 


I  )•  i>-l 


For  the  actual  delay  line  filter,  the  output  is  represented  by 


N 

y(t  }  =  - XT  ^  fcj 

eA1  j  =  0  k  5 


n 


i  =  1 


-icj(t  )  ) 

“TP  > 


{  5.28  ) 


the  variance  of  y(  t  )  is  given  respectively  by 


D 


o(  Dx(  t  )  j'fl0  |z(t,/4+  J  *  )  j 

A  la  c< 2  +  **  2 


d 


(  5.29  ) 


for  the  nonstationary  input  signal  used  for  part  B  . 

-AT 


(5.29  )  will 


If  c  is  time-invariant,  then  z(t,X  )  =  c  e 
be  reduced  to  (  5.21  )  as  it  should  be.  The  above  formulations  are  only 
applicable  to  asymptotically  stable  systems,  and  instants  which  are  suffi¬ 
ciently  remote  from  .he  initial  time  t  . 

2.  Adaptive  schemes  for  delay  line  filteis  with  slowiy  time-varying  parameters. 

V'hen  the  system  or  input  characteristics  vary  slowly  with  time,  it  is  found 

ny 

convenient  to  think  in  terms  two  time  scales  by  using  a  "fast"  time  variable  t 

A 

and  a  "slow"  time  variable  t  .  The  ratio  of  the  two  scales  :.s  a  small  number 
so  that 


A 

•  -  A  1 


( 5.30  ) 


The  "fast"  time  variable  t  refers  to  the  time  variable  in  which  the  .vlnptivf 

/V 

system  operates  while  the  "slow"  time  variable  t  reiers  to  the  time  variable  in 
which  some  parameters  in  the  system  vary.  One  example  of  the  latter  case  the 
fluctuation  of  signal  or  noise  power  levels.  The  levels  change  but  very  slowly 
so  that  nearly  all  the  techniques  developed  for  time-invariant  cases  ran  be 
applied  if  additional  modifications  are  made  to  account  for  the  effect  of  slow 
variation.  For  the  adaptive  delay  line  filters  under  study,  ir  we  know  the  forms 
of  the  signal  or  noise  correlation  functions  (  even  they  are  changing  very  slowly  ), 
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the  schemes  described  previously  can  readily  be  used.  However,  if  we  do  not 
know  exactly  how  the  correlation  functions  change  (  but  we  know  the  cause  of 
variation,  for  example,  sinusoidal  or  exponential  amp "itude  modulation,  frequency 
modulation,  etc.  )  then  we  can  assume  that  the  weight  parameters  are  functions 
of  slow  time  variables  and  leave  the  unknown  fluctuations  untouched.  In  what 
follows  the  method  of  two  time  variables^8  is  described  and  then  applied  tp 
the  tapped  delay  line  filters  with  slowly  time-varying  parameters. 

A.  Two  time  variable  method 

Consider  a  linear  system  whose  input  x(t  )  and  output  e  ( t  )  are 
relate!  by  the  differentia]  equation 

Le(t,'t)=Mx(t,t)  (5.31) 

In  the  above  equation  L  and  M  are  linear  differential  operators  and  can  be 
written  in  the  form 

L  =  L(c,t,^b)  =  <2  .  (c,  t  )  p  1  (5.32) 

i  =  0 

a  m  a  i 

M=M(c,t,4j)  =  •£  b  ( c  ,  t  )  'h  (5.33) 

j  =  0  ]  ’ 

where  the  symbol  ^  denotes  the  total  derivative  with  respect  to  time  and 
is  defined  by 
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( 5.34  ) 


Noting  that  the  adaptive  parameter  is  a  slowly  varying  function  of  the 
slow  time  variable, 


c  =  c(  t  ) 


(  5.35  ) 


V/e  can  now  expand  the  operations  L  and  M  in  Taylor  series  about  B=o 
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A  A  Ay  A 
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(  5.36  ) 


A  A  rj  a 
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=  M  (  c  ,  t  ,  p  ) 


3  -jb 
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( f3  1 


(  fli  )  +-(5-37) 
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Since  L  terminates  the  power  n  and  M  terminates  at  the  power  m  , 
(5.36  )  and  (5.37  )  can  be  rewritten  as 
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where 


J  ]! 


1  L 

j 


1  5]l0 


M  =  —  c)  1  M 
i  i  I  3T  ,  i 

•  -fl.O 


“p 

=  1  ^  M 


^  ~  o  1 

The  solution  can  be  expanded  in  the  form 


=  >  a(M0 

i  -  1  ii 


•  3 


V 


N 


e  ('tzT  )-  i  /3  e , 

j  =  o 


(  5.38  ) 


(  5.39 


(  5.40  ) 


(  5.41  ) 


(  5.42  ) 


Substituting  the  expression  of  £  ( t  ,  t  )  into  (  5.31  )  and  equating 


the  terms  with  same  power  of  p  ,  we  have 

j  N 


n  a  j  N  .  m  i  N  , 

,l0  L)<^>  £(,0  e*“  ,?oP  e‘ 


D-r.'; 


5.43  ) 


thus ,  for  j  =  0  , 


(  5.44  ) 


Lo  eo  =  Mo  x 


j  =  1 


Lo  el  =-Li  p  eo  +  Mi 


Consequently,  eQ  = 


61  = 


h 


A  (  M° 


x  )  -  M  L  j,  x'j 


( 5.45  ) 
(  5.46  ) 

( 5.47  ) 


eQ  is  just  the  solution  for  time-invariant  case  while  e  with  i  £  1 

are  additional  terms  as  a  result  of  slow  time  variation. 

A  A 

The  differentiation  with  respect  to  t  implied  by  can  be  carried  out 


explicitly. 

M 

r- 1 


)  = 


J_  f3Mo 

U  i'3  t ' 


c  =  const 


+  d  Mo  c>  c(  t  n 
9  c  B  t 


M  / 

-4- 

T  ^  ' 


o 

M 


(  3  Lo 

,  3  L0  3  c  'j 

v  a  i 

3  c  at  ^ 

c-  -  const 


o  7) 


x 


Lo  cM 

If  the  variation  in  c  is  slowly  encgh  such  that 


-N  2 

O  c  ( t  )  ^ 
..  __2 —  ^ 

a  t 


then  the  solution  has  the  !orm 

(3  ei 


A  /V 

e(t,t)^  eQ  + 


where  e^  is  obtained  by  combining 


(5.47  }  and  ( 5.48  ) 


i 

/  *  Mo 

-A  N. 

G1  ~  L 

0 

■  x  ) 

-  Mi  f  x) 

1  /  A  A  -s 

r-  iLif%  -  Mi  y  xJ 

o  ' 

7)  c(t  )  Lj  3  en  (  B  c(f )  M!  B  x 
3 ^  l0  ac  +  ^  lc  c>  c 


(  5.48  ) 


(  5.49  ) 


(  5.50  ) 
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From  Eq.  (  5.49  ),  the  mean  square  of  e  is  approximately 


2  2 
e  =  e  + 
o 


2{jeQe1  +  -- 


(  5.51  ) 


B.  Applications  on  adaptive  filters 

Let  us  now  turn  our  attention  to  the  tapped  delay  line  filter.  As  has 
been  shown  previously,  the  filter  output  is 


•  ( t  )  =  ^  c.  x  ( t  -  ) 

i=  0 


(  5.52  ) 


Using  Eq.  (  5.22  )  ,  we  can  express  Eq.  (5.52  'as 

N  N  .  _ 

?(  w)  =  y  c  x  ( u )  e  ^Ti  =  x( )  Y.  s  e  J  u  1 

i=  0  i=  0  1 

or  in  time  domain 


Zc1e->T  1 


z( t  )  =  x( t  ) 


(  5.52  ) 


Comparing  with  (5.31  ),  we  see  that 


N 

I  cie 

i=  0 


( 5.54  ) 


M  =  1  ,  unity  operator 


the  operator  is 


N 

I  a 

i=  0 


,->Ti 


(  N  _  ^ 

Z  ci e  v 


—■  (  Z  (  -  TI  >  o4  e- >  T1  ) 

i  1  i=  o  ' 


(  5.55  ) 


D-li.! 


and 


N 


Z  V, 


1=  0 


J?  -  >  T 
I  cia  ^  1 

i  =  0 


A  *p 


uv 


(  5.5  5 


Tav  defined  by  (  5,56  )  can  be  though,  as  the  average  delny  time 
of  the  filter. 

Following  (5.49  )  the  filter  output  is 
z(t  )  ~  z0(t  )  +  ^  Zj  (T,t  ) 


with 


'v  /\  jz  7s  c,(t  I 

Zj(t  ,  t  )  =  -  1  c)  iU  ] 


N  a  M 

Y  3  <M  )  c)  f  v  "l 

^  -ZTTT -  (TaV)-^-  I  c  x(t-T)J 

i=o  a  t  3  ci  S=0  ' 


N 

I 

i=  0  3 


&  ( 

t  v  L  / 


2Vi} 

3  3 


N 


'  -  l  f  x  1  :  -  T1  >  ■=  -  Tav  1|T< 1  )  1  S  (  5-57  > 

i=0  3  t  1 


where 


/  X(t  )  Nj 

)7  ( t  )  =  x(  t  -  T) 

x(t  -  tn  V 


and 


S  . 


(  L  \ 


\N  / 


(5.58  ) 

If  the  desired  signal  d(t  )  is  not  slowly  time-varying,  the  error  function 
is  then 

e  ( t  )  =  d  ( t  )  -  z  (  t  ) 


=  d( t  )  -  zn(  t  )  -  ft  z.  U  ,  t  ) 


f3 


=  e  + 
o 


I3  h 


(  5.59  ) 
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The  mean  square  error  is  approximately 


e2(t  )  £  ec2  +  2  |g  e0e 


1 


( 5.60  ) 


Or 


ez(t  )  =  F. 


[  (d(t)-  T  ci  »[i  <t0  | 

r  /  N  -  T ,  ) 

+  2  p  E  J  (d  -  Z  pj  (t  )HT  rj(t )  &  )  j 


N 


=  E  \  [  d(t  )  -  J  ci 
i=  0 


C  N  T  'V 

+  2pTav  E  |  ^(DjjjjU))  £  (5 


Taking  the  partial  derivative  of  e2  with  r'spect  to  ,  we  have 


.61  ) 


2> 


N 


T, 


-  2  ,3  v  E  J  *7 i(t )  <*  > 


Thus  the  gradient  of  e4  is 
E 


jyc  c  (X|  c)|  =  -2  E  |  j|  ( t  )  (  d  ( t  )  -  ^(t  ) :  c  | 


"2  p  Tav  E 


(t  )  n( t  ) 


or 


'/c  Q<- 


c  )  «  -  2  n(t  )  f  d(t  )  -  rj(t  )  c  ^ 

- 2  V.  >1"  »  ‘-h 


( 5.62  ) 
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Using  the  adaptive  schene 


£j+  i  =  "  Tj  Vc  I  ' 

we  obtain  a  new  scheme  for  the  time-varying  case 

T 

£  j+i  -  £  j+  2rid,  r!,-2^)  V  'I  j  -  , 

=  s  /  2r(  |jV  2r,pav-j4Ti  <5- 

Algorithm  (5.63  )  can  readily  be  implemented  by 


"M  Ct’ 


Thus  the  increment  for  the  time-varying  parameter  c.  is  obtained. 

Similar  schem--.:  can  be  obia.ned  through  proper  trar:  .formations ,  Eqs . 

(  4.42  )  and  (  4.44  ),  fe:  the  cases  where  only  the  signal  or  the  noise  corre 
iation  functions  are  assumed  to  be  known.  The  results  are 

-Vl  "V  2  A'j  p  Tav  7  , 

-2r>  h  2  r,  'i,  %  ’ 

when  1^  (  7”  )  is  known,  and 

%.  ’V  2MT-  7  ,  1*6 

-  2Ti  f?  ,  n  +  2  K,  P 


3.  Effects  of  slow  time  variation  on  minimum  mean  square  error. 


n 

If  we  set  ^7  c  =  0  ,  we  see  from  Eq ,  (5.61  )  that 

2  rj  rj  T  c*  =2i]  d  +  2  p  Tavt]  )i  T  $ 

—  =  d  n  +  j6  Tav  ~rj  ^ 


or  c 

Rn 

d  )o 

+ 

8  Tav 

1 

1 

l 

"k 

Substituting  the  expression  of  c  for  c  into  (  r'.61  )  and 
using  (  3.24  b  ),  we  get  the  minimum  mean  square  error 


e2mi„  =  d2  -  d 

^  T  * 

1  ~ 

+  2pav 

Ej  |  TfJ 

(  d  (  t  )  - 

r  C  ! 

e2  .  =  d2  -  d  /? 

min  1 

T 

*1 

1 

d  rj 

-  p  Tav  d  -}  T  i 

T 

T 

"I 

1  d  Ij  +  p  Tav  S 

) 

T  _ 

— 

+  2  pav 

S  d 

1 

X5 

1 

CM 

~o 

II 

T 

1  «-) 

- 1 

d  n 

-  B  Tav  S’  "I 

CM 

CD _ 

CM 

1 

T  2  .  T 
Tav  ,) 

Y  s 

'  1 

C  2 

p  4- 

~~  °  min 

P  Tav  | 

T - 

d  ^ 

2  2  T 

+  2  (3  Tav  b  Rj 

S 

(  5.67  ) 

2  h2 

wnei  e  =  d 

o  min 

T 

'  ’( 

-  i 

d  tj  is  the  minimum 

T 

mean  square  error  of  the  time -invariant  i i Iter  as  derived  in  (  3.24  a 

). 

The  expressions  e  2  m ,  n  for  other  eases  (known  R,-,  ('JJ  )  or 
RS(T  )  )  as  well  as  the  erfect  of  sic-.  time  variation  on  me  rate  oi  conver¬ 


gence  can  be  obtained 


fr;i  i  pi¬ 


rns'  ton 


Appendix  A 


Proof  of  the  modified  algorithm 

It  was  mentioned  in  chapter  IV  that  algorithm  (4,7)  and  (4.11)  were  derived 
from  the  formula 

cj+1  -  Cj  -  Yj  (7Q:  +  7Q2)  (A • 1) 

rather  than  from 


cj+1  =  c.  -  Yj  (VQX  +  VQ2)  (A. 2) 

where  +  Q2  =  q  is  a  function  of  error,  and  the  average  of  it  is  the  perform¬ 
ance  criterion  to  he  minimized. 

Comparing  (A.l)  and  (A. 2)  with  the  regular  gradient  method  with  constant  y 


CJ+1  ■  Sj  -  V  (VQj.  +  vq2) 


(A. 3) 


we  see  that  in  (A. 2)  no  average  is  taken  while  in  (A.l)  partial  average  is  taken. 
The  error  £  caused  by  measurement 
VQ  =  VQ  +  ? 

is  eliminated  by  the  properly  chosen  sequence  {y^}  .  Intuitively  speaking,  the 

same  {y_.}  which  eliminates  the  error  caused  by  (VQ  +  VQ  )  minus  (VQ  +  VQ0) 
j  X  /  X  t- 

can  definitely  eliminate  that  caused  by  (VQ^  +  minus  (VQ^  +  VQ^) .  This 

stems  from  the  fact  that  the  measuring  noise  in  the  second  case  is  smaller  on 
the  average  than  in  the  first  case.  Although  Intuition  dees  not  generally  warrant 
mathematical  correctness,  we  can  state  with  mathematical  rigour  that  either  signal 
or  noise  statistical  properties  will  suffice  to  generate  the  error  gradient  used 
in  the  adaptive  schemes.  The  physical  conditions  under  which  these  algorithms 
converge  remain  unchanged.  Two  lemmas  and  one  theorem  will  be  proved  in  sequence. 
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Lemma  1, 


For  the  tapped  delay  line  filters  considered  in  Chapter  4,  if 
(n)  Q  -  Q3  +  Q2 

(b)  Q(e)  =  e2 

(c)  7Q7  is  independent  r  __  , 

r  i "  i  Ties 

(^ql  + 

c  =  c* 


(A. 5) 

for  all  1  (A. 6) 

and 

T  "J 


inf 

r  <  |  |  £  - 

c* !  |  <  - 

—  f 

E  ; 

(c  -  c*) ' (VQ 

i +  v;-'  ° 

/ 

(A.  7) 

E  >  0 

Since  VQ^  is  independent 

oil  c  , 

we  have 

E  S, 

(c  -  c*  :  T 

(VQp  +  ' 

'V, 

=  E  1  (c 

-  r_*>T  "Qp 

!  + 

,  (c  -  *)  ; 

vqJ. 

=  E '  (c 

-  c*)T  VQ, 

1  +  e 

T 

,  (£  -  £*) 

E  '•’<£ 

=  E  j  (c 

-  c*K  (VQ 

,  +  -V 

(A. 8) 

Therefore,  by  virtue  of  (A. 

7) 

inf 

E  <  II  C  - 

c* 1  I  <  “ 

p  .  o 

-  r*)  '  ('(}, 

+  vq  v,  >  n 

(A.  9) 

e  >  0 


then  at  the  neighborhood  of  c*  wM  ~h 


statement  is  true: 
inf 

E  < 


c  -  c*  1  i 


E  '  (c  -  c*) 
t  “  ~ 


E  >  0 


Proof:  If  Q  =  Qj  +  has  a  minimum  at 


o(Q1+Q2  >  0  for  c.  •  c  * 

-  i  l 


3c , 


-  0  for  c  =■  c^* 

<  0  for  Cj  <  Cj* 


tiius 


b(Q,  +  Q . ) 


T  ( t  ' 


the  following 


VQJ  '  0 


(A.  4) 


,  then 


I  f  7  r 


Lemma  2 


If 


Q  «=  ,  Q(e)  =>  e  ,  VQ?  is  independent  of  c_  , 


and 


(a)  3  Q 


_1  exists  and  is  uniformly  bounded 


3e 

(b)  s(c)  and  n(t)  are  uniformly  bounded, 
then  for  the  tapped  delay  line  filter 

E  j  (VQ1  +  VQ2)T  (VQ1  +  VQ2)|  <  d(c*Tc*  +  cTc) 


d  >  0 

Proof : 

Using  a  Taylor  expansion  about  £  =  c*  ,  we  have 


3QX  3Qj 

3c.  3c, 

j  j 


N 


32Q. 


+  i=o  (Ci  "  ^  Sci"ci 


c  =  c* 


for  arbitrary  j  ,  and  j  =  0  ,  1,  2,  - ,  N 

Since  e(t)  =  s(t)  -  z(t) 

N 

■-  s  (t)  -  2  c  n.  (t) 
i=0 

we  see  that 


3Q1(e) 

3c , 


3Q  3e  3Q 


and 


3  Qj/c) 

3c , 3c . 
i  J 


3  2Qi 


3e 


ni(t)  Pj(t) 


3  2Qi 


3e 


[s(t  -  T^)  +  n(t  -  T^)  ]  [s(t  -  Tj)  +  r. (t  -  )  ] 


(A. 10) 


(A. 11) 


(A. 12) 


(A. 13) 


(A. 14) 
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Thus 


32Q. 


3ci3cj 


is  bounded  if  (a)  and  (b)  are  s  it:'  ■  f  d 


Therefore,  from  (A. 11) 


Vc  Q1  *  Vc  *1 


N 


+  k. 


]  1  (ci  “  cl*) 

c  =  c*  i=0 


(A. 15) 


where  k-.  =  k  sup 

1  all  i 


9ciScj 


(A. 16) 


As  Q2  contains  c_  only  in  the  first  order  and  V  is  Independent  of  c_  , 
we  can  write 


V2  -  VcQ2 


c  =  c* 


and 


VQ:  +  VQ2  <  7Q1 


N 


+  VQ,,  +  k,  1  (c  -  c  *) 

|c  =  c*  ]c  =  c*  1-0 


fA .  1 7) 


Note 


E>  VQl 


VO  -  ^  = 


'  —  \  I 

+  VQ,  1  =  E  )  VQ  +  :c,  1 
c  =  c*  j  1  v  —  c* 


=  E  \  7Q:  +  VQ2  ) 


0 


|c.  =  c* 

Taking  mathematical  expectation  on  both  sides  of  (A. I/)  gives 


(A. 19) 


f 


N 


E  j  7Q  +  VQ  (  <  k  E  (c  -  c  *) 

1  1  1 J  1  1=0  1  1 

Lemma  2  is  obtained  by  taking  the  inner  produet  o:  (A. 20) 

E  |  (7Q1  +  VQ2)  (7Q1  +  VQ2)1 
2  N  N 

<  k,  l  l  (c .  -  c  .  * )  ( c  -  * 

“  1  i=0  j=0  1  1  ' 


(A. 20) 


d(c*^  c*  +  cr  c) 


(A. 21) 
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Theorem 


Let  <  Y2  "  "  ”  be  a  sequence  of  positive  numbers  such  that 


(Al)  lin  v  =  0 

j+oo  J 


(A2)  E  y.  =  03 
j=0  J 


(A3)  E  Y.  <  00 
j=l  J 

Let  the  following  conditions  be  satisfied 


inf 


'N 


E  (c  -  c*)  (V  Q  +  VQJ  \  >  0 

(B>  e  <  ||c  -  c*||  <  -  1  ' 

,  e  >  0 

(C)  E  \  (VcQ1  +  7Q2)T  (7Qx  +^Q2)  ( 

<  d  (c*T  c*  +  cT  c)  , 


d  >  0  ,  for  all  c.  in  a  bounded  set. 
Then  the  algorithm 


-j+i  ’  -3 '  b  <VQi +  'V 

which  minimizes  the  performance  criterion 

">  \ 

1(c)  =  E  ,  Q(e)  |  =  E  j  q,  (e)  +  Q,(e) 


v. 


<  1  V  /  ■  x  2  ' 

converges  with  probability  and  to  c*  . 

Proof : 

bubstratlug  both  sides  of  (A. 26)  by  c*  ,  we  have 
£j+1  -  £*  =  c  -  c*  -  Yj  (VQj.  +  VQ2) 
taking  the  inner  product  on  both  sides  of  (A. 23) 

(— j+i  -  ^*)T  (^j+l  -  £*> 

=  (£j  -  c*)T  (Cj  -  c*)  -  2  Yj(c,  -  c*)1  (VQX  +  VQ?_) 


Y  2  (vqx  +  pq2)t  (vq1  +  vq2) 
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(A. 23) 


(4.24) 


(A. 25) 


(A. 26) 


(A. 27) 


(A. 28) 


(A. 29) 


and  taking  the  conditional  mathematical  expectation  for  given  , 


c^  ,  we  obtain 


E  -  c..,  -  c* 


-J+1 


-1  *  -2 


M 


=  I  |cj  -  c*|  |Z  -  2Yj  E  \  (Cj  -  c*)‘  (VQ 
+  Yj2  E  |  (VQ1  +  VQ2)T  (7Qx  +  VQ2) 


i +  'v'l 


From  condition  (c) ,  (A. 30)  becomes 


E  \  lie 


l  1 '“j+1  - 


^  i 


\  '  -T 


vn  ’ll 


<  ||Cj  -  c*||  -  2Yj  E  |(£j  -  c*)  (7Q1  +  vq2)j 


2  T  T 

+  Yj  d(c*  c*  +  c^  Cj) 


Using  condition  (B) ,  (A. 31)  is  reduced  to 


E  1  I  I  cj+JL  -  c*  |  | 


/ 

c  \ 
— n  \ 


<  I  kj  -  c*|  |‘  (1  +  2y^2  d)  +  2Yj2  d  CT  c* 

Using  condition  (B) ,  we  can  reduce  (A. 31a)  to 


E  |  I  I £j+1  “  £*l 


i  2 


—  >^n) 


<  I  !  Cj  -  c*||2  (1  +  2Yj2  d)  +  2Yj2  d  CT  c* 


Let 


d  =  I  \±A  -  £*i  I  2  71  d  +  Yk2  d) 

J  J  k=j 


+  I  2d  Y 
k=j 


2  T 

k  i 


ra=k+l 


(i  +  Y  d) 

m 


Then 


d+1  =  l|c,+1  -  c*||2  .  (1  +  Yk2d) 

-j+1  j+1  k=j+1  k 


+  Z  2d  Y  2  c^  c*  77  (1  +  Y  d) 

k=j+l  m=k+l 


(A. 30) 


(A. 31) 


(A. 31a) 


(A. 32) 


(A. 33) 


(A. 34) 


1>7! 


Taking  the  conditional  mathematical  expectation  for  given  c,  ,  c„,  -  c.  , 

— 1  — 2  — 1 


we  have 

r 


E !  -j+i|c 


1  ”  cn  ■:  =  E  ) 


lcj+l  -  c*||‘ 


00 

»--  O  71  (1  +  Yv2  d) 


k=j+l 


-f  1'  2  d  y .  “  cT  c*  -n  (1  +  y  2  d) 

k=j+i  k  m=k+l 


-  r* 


i  f  ^ <  1  +  d  y  2)  +  i  y/  d  c‘  c*  |  7i  (1  x  y.  4  d) 
1  3  '  k=j+l 


+  L  2dy  2  c.1  c*  7t  (1  +  y  2  d) 
k=j+l  m=k+i 


=  h 


or 


E)  2. 


-j+1 


:l’""’Cn  t  -  -7-j 


(A. 35) 


Next  taking  the  conditional  mathematical  expectation  for  given  ,  - ,  Z 

betn  sides  of  (A. 35)  we  have 


on 


E'  V  i”i’ 


(A.  36) 


Inequality  (A, 36)  shows  that  is  a  sea.imart ingale,  where 


E 


/  <  <» 
-1 


(A. 37) 


22 


so  that,  according  to  the  theory  of  sen tmartingales  the  sequence  Z^  converges 
with  probability  one,  and  hence  by  virtue  of  '.’'>.33)  and  (A.  23c)  the  sequence 


(Cj  -  £*)  also  converges  with  probability  one  .:o  son.:  random  number  £  .  It 
remains  to  show  that  P(t,  =  0)  =  1. 


It  is  seen  that  from  (A.  37),  (A. 35)  and  (A .  23«  >  tr.e  sequence  E(c^  -  c*)  is 
bounded.  Now  taking  the  mathematical  expectation  on  both  side  cf  the  inequality 
(A. 32), 


E  :  I'vi  -t*H2  )<  * 


1  Hi,  -  c*||2  j  -  2,.  F.  \  (c,  -  c*)TV0  1 


+  Yj2  d  (C*T  c‘  +  2<c3T  V1 
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and  adding  the  first  j  inequalities  together,  we  have  by  deduction 

i  r  v:  J 

r 

k“l 


l<Lj+1  "  £*|  |  j  <  E  ^  ||c 


1  -  c*|  |2(t  r  [c*T  c*  Y|,2  +  d  Yk2  E(c^T  c)] 


j 


-  I  2Y  E  (c  -  c*)  VQ 
k=l  K  k  J 

Since  E  j  |  J -  c*||2(  is  bounded  ana  condition  (A. 23c)  is  fulfilled,  from 
Eq.  (A. 38)  it  follows  that 
» 

1  y,  E  i  (c  -  c*)T  VQ  )  <  » 
k=l  k  J 

00 

Using  condition  (A. 23b),  i.e.,  I  Y.  “  ” 
and  noting  (A. 24) 


(A. 38) 


(A. 39) 


inf 

e  <  I  I  c  -  c*  I  I  < 


E  (c  -  c*)  VQU  0 


We  deduce  from  (A. 39)  that 

(c^  -  c*)T  VQ -►  0  with  probability  one  lor  some  sequence  N  (A. 40  • 

/  . .  .  O  / 

Now  taking  E  j  |  |  c_.  -  c*  '  \  with  probability  1,  and  comparing  (A. 40)  with 

(A. 24) ,  we  obtain 

£=  C  with  probability  1  (A. 41) 

Therefore,  algorithm  (A.  26)  converges  v.:  '„  probability  or.e 


Pi  lira  (c,  -  c*)  =0 

as  well  as  in  mean  square  sense,  i.e., 
lim 


j- 


(A. 42) 


(A. 43) 
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Appendix  B 


Some  properties  of  Gamma  functions 

Since  T(a  +  n)  =  (a  +  n  -  1)  T(a  +  n  -1) 

=  (a  -1-  n  -  1)  (a  +  n  -  2)T(a  +  n  -  2) 

=  (a  +  n  -  1)  (a  +  n  -  2) - af'Ca'' 


We  have 


TJ  ('■.  +  k  -  1)  =  a(a  +  1) - (a  +  n  -i) 

k=l 

T (a  +  n) 

T(a  ) 


Thus  Eq.  (4.29)  becomes 


j 


TI  (i  ~  TTT)  =  ’f  (j+1  -*) 
k=l  J  _k^l _  T(i+2  -X 


C+1)! 


(j+1)!  T(2  ->.) 


Eq.(B.2)  can  be  approximated  by  using  the  formula* 


TOO  =  2* 


1_ 

x-  2 

i 

571 


2488320  x 


-  1  1 
(2-)  |  12x  28gx2 

-  +  0  (  iT> 


139 


51840x' 


-x 

C  X 


1 

X-  2 


( 2tt )  for  x  >>  1 
From  Eq.  (B.3)  we  can  write  for  j  >>  1  , 

r(j  +  2  -  a)  -  e'(j+2“a)  (j+2-a)  j+2_a_  2  (2rr) 


=  G 


3  1 

■  ( j+2-a)  (j+2_a)j+  2  (j+2-a)"a  (  2tt  )  2 


*  Whittaker  and  Waston,  Modern  Analysis,  p.  253 


(B.l) 


( B .  2 ) 


(B.3) 


(B.4) 
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(j+1)  i 


r(j+2) 


(B.  5) 


?  e'(j+2)  (j+2)J+  2  ( 2ti )  2 

Since 

j  +  2  -a  =  j+2  if  j  >>  a 

we  obtain  from  (B.4)  and  (B.5) 


£ (1+2-g) 

(j+D  ! 


1 (1+2-g) 

r  (j+2) 


(j+2-a)' 


(J+1) 


if  j  >>  1  and  j  >>  a 


Therefore,  combi~ing  (B.2)  and  (B.6)  gives 


j 


n  (1  "  T-T7 

k-1  jH 


r (2-X) < j+3 ) 


si 


and  furthermore, 
m 

t  (1  - 
j  =  l 


j+1 


(n+l)'‘ 


(B.6) 


(B.  7) 


(B .  c] 
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